-- Logs begin at Mon 2022-07-18 15:49:03 UTC, end at Mon 2022-07-18 17:01:07 UTC. -- Jul 18 16:22:32.981296 dv-cinder-837543-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 16:22:32.994238 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 16:22:32 2022] *** Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: nodename: dv-cinder-837543-6-iscsi Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: machine: x86_64 Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: clock source: unix Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: pcre jit disabled Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: detected number of CPU cores: 4 Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: current working directory: / Jul 18 16:22:32.997394 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: detected binary path: /usr/bin/uwsgi-core Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: your processes number limit is 47680 Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: your memory page size is 4096 bytes Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: detected max file descriptor number: 2048 Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: lock engine: pthread robust mutexes Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: thunder lock: enabled Jul 18 16:22:32.997974 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 16:22:32.998289 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 16:22:33.040387 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: Python main interpreter initialized at 0x558e0d4fe9c0 Jul 18 16:22:33.040387 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: python threads support enabled Jul 18 16:22:33.040387 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: your server socket listen backlog is limited to 100 connections Jul 18 16:22:33.040387 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: your mercy for graceful operations on workers is 90 seconds Jul 18 16:22:33.040867 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 16:22:33.040867 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: *** Operational MODE: preforking *** Jul 18 16:22:33.041185 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: *** uWSGI is running in multiple interpreter mode *** Jul 18 16:22:33.041185 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: spawned uWSGI master process (pid: 80154) Jul 18 16:22:33.041440 dv-cinder-837543-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 16:22:33.044174 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 1 (pid: 80165, cores: 1) Jul 18 16:22:33.044174 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 2 (pid: 80166, cores: 1) Jul 18 16:22:33.044174 dv-cinder-837543-6-iscsi devstack@keystone.service[80154]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 16:22:38.768630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:22:38.768630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes Jul 18 16:22:38.769487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:22:38.769487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes Jul 18 16:22:38.821802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:22:38.822043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:22:38.822199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] command line args: [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:22:38.822531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:22:38.822700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ================================================================================ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:22:38.822970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.823227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_dir = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.823421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.823609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_source = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.823793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] debug = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.823999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.824406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.824573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.824819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.825079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.825313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.825512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_config_append = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.825672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.825852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_dir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.826025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.826272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.826556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.826810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.827061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.827241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.827425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.827675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.827921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.828162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.828398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.828638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.828873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.829125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.829457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.829740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.830285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.77/identity {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.830485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] publish_errors = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.830729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.830963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.831402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.831652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.831893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.832133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] standard_threads = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.832366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.832617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.832795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.832975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_journal = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.833155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_json = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.833334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_stderr = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.833513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_syslog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.833746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.834009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.834252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.834552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.834799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.835060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.835305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.835557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.835797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.836145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.836348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.836576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.836835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.837113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.837375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.837634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.837903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.838147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.838421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.838690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.838943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.839185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.839444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.839713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.839976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.840232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.840468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.840703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.840938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.841177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.841433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.841712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.841980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.842214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.842494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.842759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.843025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.843293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.843553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.843809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.844073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.844321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.844578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.844825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.845612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.845869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.846147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.846395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.846683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.846931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.847193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.847445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.847720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.847966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.848230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.848482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.848744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.849007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.849257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.849503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.849779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.850041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.850296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.850570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.850858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.851123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.851369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.851623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.851884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.852189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.852441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.852705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.852982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.853241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.853498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.853751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.854025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.854278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.854576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.854841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.855134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.855397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.855669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.855946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.856217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.856527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.856806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.857089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.857362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.857646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.857920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.858196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.858493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.858768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.859036 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.859359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.859632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.860007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.860298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.861227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.861496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.861743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.862033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.862305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.862623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.862864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.external = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.863127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.863456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.863712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.864002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.password = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.864226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.token = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.864493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.864735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.864988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.865225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.865479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.865753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.866019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.866267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.866619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.866882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.867153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.867418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.867672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.867913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.868202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.868481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.868743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.869004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.869329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.869600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.869889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.870140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.870435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.870672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.870914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.871164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.871417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.871653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.871920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.872184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.872445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.872722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.872991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.873245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.873501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.873743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.873998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.874271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.874559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.874802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.875077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.875331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.875579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.876096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.876096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.876397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.876638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.876892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.877164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:22:38.877164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes Jul 18 16:22:38.878124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:22:38.878618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes Jul 18 16:22:38.878933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.878933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.879670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.879670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.879670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.880137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.880322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.880517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.880787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.881052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.881328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.881594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.881859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.882135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.882474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.882683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.882947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.883206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.883470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.883718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.884011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.884259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.884520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.884779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.885038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.885278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.885516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.885771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.886024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.886266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.886568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.886826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.887068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.887475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.887626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.887858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.888130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.888383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.888642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.888884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.889117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.889357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.889596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.889838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.890072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.890333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.890614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.890865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.891117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.891365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.891600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.891894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.892138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.892379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.892620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.892858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.893109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.893362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.893605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.893865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.894101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.894353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.894629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.894869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.895109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.895361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.895611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.895848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.896090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.896334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.896590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.896844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.897084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.897385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.897596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.897835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.898085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.898327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.898599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.898838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.899089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.899324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.899579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.899831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.900090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.900347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.900598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.900846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.901096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.901346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.901584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.901837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.902089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.902342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.902627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.903070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.903314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.903570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.903825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.904079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.904334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.904574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.904837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.905096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.905340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.905577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.905826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.906064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.906321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.906611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.906864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.907105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.907345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.908136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.908389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.908633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.908889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.909130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.909390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.909641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.909885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.910134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.910382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.910660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.910909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.911153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.911392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.911643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.911899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.912133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.912380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.912681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.912948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.913199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.913451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.913693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.913939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.914178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.914438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.914681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.914920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.915169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.915409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.915655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.915910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.916162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.916401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.916651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.916886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.917147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.917388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.917548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:22:38.950643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.950883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.951122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.961039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.968587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:22:38.968925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:22:38.969282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] command line args: [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:22:38.969786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:22:38.970655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ================================================================================ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:22:38.971126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.971583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_dir = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.971931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.972272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_source = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.972617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] debug = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.973029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.973496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.973840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.974287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.974780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.975231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.975576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_config_append = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.975910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.976246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_dir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.976581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.977042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.977515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.977884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.978292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.978610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.978876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.979233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.980162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.980629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.981077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.981533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.981982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.982499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.982939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.983435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.984343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.77/identity {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.984688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] publish_errors = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.985135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.985579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.986491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.986960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.987424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.987856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] standard_threads = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.988297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.988769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.989100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.989435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_journal = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.989764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_json = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.990117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_stderr = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.990489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_syslog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.990949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:22:38.991444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.991903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.992403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.992856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.993259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.993401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.993535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:38.993764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.994251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.994750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.995245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.995718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.996193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.996642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.997105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.997555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.998024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.998534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.998983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.999470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:38.999961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.000406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.000841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.001402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.001895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.002377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.002890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.003331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.003776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.004215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.004656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.005136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.005603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.006043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.006519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.006997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.007461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.007918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.008374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.008825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.009269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.009720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.010172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.010678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.011074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.011468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.011869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.012259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.012661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.013055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.013440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.013807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.014176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.014618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.014979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.015368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.015756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.016158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.016577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.017053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.017509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.017979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.018297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.018699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.019040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.019423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.019680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.020090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.020409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.020663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.020952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.021237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.021610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.021918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.022172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.022609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.022824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.023085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.023364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.023635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.023905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.024161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.024462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.024798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.025156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.025500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.025785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.026083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.026436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.026744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.027057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.027359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.027635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.028048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.028477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.028982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.029237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.029637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.030076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.030578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.031034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.031517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.031961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.032961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.033478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.033943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.034382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.external = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.034845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.035401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.035826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.036270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.password = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.036708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.token = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.037172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.037619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.038095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.038539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.038987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.039448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.039878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.040322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.040808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.041279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.041752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.042210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.042712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.043163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.043920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.044155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.044621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.045112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.045652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.046135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.046668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.047134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.047618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.048096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.049092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.049570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.050030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.050510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.050910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.051336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.051814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.052253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.052725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.053180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.053648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.054094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.054579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.055055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.055508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.055956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.056404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.056834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.057290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.057763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.058243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.058721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.059155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.059604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.060073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.060555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.061017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.061515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.061988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.062537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.062988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.063438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.063902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.064342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.064785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.065232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.065700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.066156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.066653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.067080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.067551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.068018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.068466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.068940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.069405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.069822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.070275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.070711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.071155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.071551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.072017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.072455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.072895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.073372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.073840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.074276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.074748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.075223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.075700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.076150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.076605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.077061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.077503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.077971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.078498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.078921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.079916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.080352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.080827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.081528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.081945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.082390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.082852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.083296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.083719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.084166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.084619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.085078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.085542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.086015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.086467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.086932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.087378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.087818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.088265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.088719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.089182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.089669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.090115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.090624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.091072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.091534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.091979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.092416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.092851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.093312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.093786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.094218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.094700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.095810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.096287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.096767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.097214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.097692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.098164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.098648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.099118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.099531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.099963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.100360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.100818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.101255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.101719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.102357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.102856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.103278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.103746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.104212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.104678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.105155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.105586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.106040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.106546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.107016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.107478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.108264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.108708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.109183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.109654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.110113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.110659 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.111109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.111600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.112066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.112513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.112951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.113399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.113838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.114320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.114858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.115325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.115790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.116235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.116648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.117062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.117498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.117974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.118463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.118947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.119413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.119862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.120329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.120838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.121320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.121763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.122214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.122698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.123138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.123609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.124035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.124492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.125039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.125562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.125984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.126538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.126986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.127438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.127885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.128337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.128784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.129245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.129724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.130182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.130695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.131159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.131638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.132079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.132553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.132986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.133441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.133884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:22:39.134200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:22:39.140945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.184976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.185361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.185751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.193877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.194110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.202848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.240273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.240663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.265287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.265642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.265762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.265939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.266163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.266384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.294123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.294368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.294632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.294858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.295079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.295297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.295515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.307723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.312924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.321491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.321817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.321976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.322197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.322462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.322683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.322901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.341627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.341842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:22:39.342173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.342377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:22:39.342687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.343011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.343212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.343474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.355838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.356137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.356498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.356789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.365488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.369153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.369466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.369761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.370188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.370516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.370789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.371071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.371356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.371635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.371919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.372196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.400823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.402614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.402908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.409144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.409446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.409734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.417480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.417780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.422970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.423589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.425541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.445575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.445809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.450669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.455214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.455541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.461270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.465214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.469354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.469675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.470007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.470317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.470636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.470928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.490806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.491242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.491523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.491800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.500251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.502736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.503055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.507249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.507536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.507835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.508130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.508424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.511480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.511878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.523773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.524146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.524459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.524765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.525064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.525395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.525698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.546113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.549593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.553116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.555278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.557078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.557364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.557657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.566074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.567816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.571494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.571805 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.572095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.572380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.582880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.583187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.583490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.591844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.592205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.592472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.592753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.593058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.593350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.593630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.593914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.594027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.594194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.594309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.594495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.594613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.594779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.594906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.595087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.595295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.595410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.595621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.595748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.620509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.620907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.621303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.621693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.622072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.622498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.622888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.635429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558e0d4fe9c0 pid: 80165 (default app) Jul 18 16:22:39.639293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.639488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b6f57-2c92-40de-a8de-912b362e0b55 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:39.639648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b6f57-2c92-40de-a8de-912b362e0b55 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:39.639882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b6f57-2c92-40de-a8de-912b362e0b55 None None] PATH_INFO: `/v3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:39.645056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1/1] 10.222.0.77 () {50 vars in 791 bytes} [Mon Jul 18 16:22:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:22:39.645389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.645778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.646741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.646741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.647786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.647786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.647786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:22:39.668562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.668912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:22:39.669515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.669847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:22:39.670323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.670843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.671308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.671762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.693652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.694063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.694486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.694906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.714639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.721692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.722186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.722718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.723216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.723753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.724242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.724739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.725225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.725692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.726168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.726678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.782828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.786847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.787779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.804253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.805180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.805922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.825006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.825522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.834681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.835110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.835485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.835863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.836227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.836603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.836972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.837335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.875743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.876385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.886677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.894790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.895325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.905523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.913650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.921075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.921617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.922153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.922692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.923240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.923880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.963209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.963718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.964184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.964702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.980944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.984504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.984971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.991263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:39.991683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.992165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.992660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:39.993141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.020386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.020897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.021321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.021797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.022290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.022767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.023249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.057843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.062892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.069598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.072686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.075949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.076451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.076949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.092052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.095819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.102295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.102796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.103282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.103660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.121218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.121728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.122155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.138011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:22:40.138501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.138961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.139447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.139936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.140431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.140902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.141348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.142545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.143043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.143541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.144030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.144501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.144960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:22:40.221326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558e0d4fe9c0 pid: 80166 (default app) Jul 18 16:22:47.739108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:47.739841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:47.739841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:47.741129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 2/2] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:47.750702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:47.751779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:47.751779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:47.821217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: INFO dbcounter [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] Registered counter for database keystone Jul 18 16:22:47.844014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG oslo_db.sqlalchemy.engines [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:22:47.851696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writer thread running {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:22:48.044142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:22:48.044656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:22:48.103778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:48.105441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:48.245480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1/3] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:22:48.428255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:48.500583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: INFO dbcounter [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] Registered counter for database keystone Jul 18 16:22:48.523572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG oslo_db.sqlalchemy.engines [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80165) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:22:48.530953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writer thread running {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:22:48.679261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e0a6b-212b-41e1-a1a4-017bcddab213 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:48.683666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:48.684371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:48.684661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:48.684944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:48.695249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:48.749988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:22:48.750683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3e4f3-cc12-4573-87f8-18c3ddc3978b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:48.762584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 3/4] 10.222.0.77 () {58 vars in 1324 bytes} [Mon Jul 18 16:22:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:22:48.769955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:48.791532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220e9ac-f7db-4aa1-b26a-f93196d62109 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:48.798883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:48.800351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:48.800833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:48.801283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] PATH_INFO: `/v3/projects/c66249a1b75d4939b69e5011863c8709` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:48.803424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] RBAC: Authorizing `identity:get_project(project_id=c66249a1b75d4939b69e5011863c8709)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:48.874000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:22:48.874551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd07467-915f-4bf4-842b-816924d51a1e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:48.880407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 2/5] 10.222.0.77 () {56 vars in 1207 bytes} [Mon Jul 18 16:22:48 2022] GET /identity/v3/projects/c66249a1b75d4939b69e5011863c8709 => generated 344 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:22:51.021902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370fc32-ed1f-4ff6-a68b-113268fa9b1a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:51.022483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370fc32-ed1f-4ff6-a68b-113268fa9b1a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:51.022616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370fc32-ed1f-4ff6-a68b-113268fa9b1a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:51.026965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 4/6] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:22:51.034029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:51.034633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:51.034633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:51.055470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:51.057090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:51.065505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 3/7] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:22:51.089432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370fc32-ed1f-4ff6-a68b-113268fa9b1a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:51.102637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370fc32-ed1f-4ff6-a68b-113268fa9b1a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:51.107068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:51.107739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:51.108030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:51.108312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:51.118042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:51.119943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15157f-a446-4d4d-a093-6c89ca30f2c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:51.130995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 5/8] 10.222.0.77 () {58 vars in 1324 bytes} [Mon Jul 18 16:22:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:22:51.140718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:51.157734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb6a66-1dba-4fd3-b22e-f1062b289d90 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:51.164679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:51.166760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:51.167460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:51.168268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:51.171532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:51.175903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9abdf-7be9-451e-a58d-b5c03ea9a114 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:51.179594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 4/9] 10.222.0.77 () {56 vars in 1198 bytes} [Mon Jul 18 16:22:51 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:22:53.477342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988815e8-3009-4f41-b5cc-bd03eea93ae4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:53.478169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988815e8-3009-4f41-b5cc-bd03eea93ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:53.478387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988815e8-3009-4f41-b5cc-bd03eea93ae4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:53.479971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 6/10] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:53.490076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:53.490670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:53.491138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:53.519871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:53.521914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:53.564414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0d8a-36b0-4696-b813-b18604e31373 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:53.564781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0d8a-36b0-4696-b813-b18604e31373 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:53.565123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0d8a-36b0-4696-b813-b18604e31373 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:53.566686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 7/11] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:22:53.572957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:53.573330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:53.573675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:53.619632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80165) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:22:53.620193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80165) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:22:53.652966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 5/12] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:53.668407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:53.670110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:53.677704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:53.708585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:53.815626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d38ad0-a449-494f-bc5c-f2d36c42aa83 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:53.822848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 8/13] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:22:53.849564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:53.861222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:53.862846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:53.863411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:53.863952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:53.883687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:22:53.901776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:22:53.903186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:22:53.904533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 6/14] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:22:53.914470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:53.967462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7e14b-6e14-4733-b3cb-f577ecaa4b75 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:53.970204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.015606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.017271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.017684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.018078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.070004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.099512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.101087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.104341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250630d-656a-4fc9-8d03-f63421c105d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.169922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.171144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.171648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.172144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.174158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:22:54.174756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.177492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.199379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 7/15] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:22:54.205585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:54.213236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 9/16] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:22:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:22:54.219125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:54.223087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.365972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d75bdb-22d4-4acb-8200-8664133424c5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.366652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f9b61-222e-4012-871f-02a28ba7d4c7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.410421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.411502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.411724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.411901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.412221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.412523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.412804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.413187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.413704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.415431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.417407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 8/17] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:22:54 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:22:54.423828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:54.437055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.439187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.442207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:22:54.465883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:22:54.467587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:22:54.468764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:22:54.468764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:22:54.468764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:22:54.468764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:22:54.468764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 10/18] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:22:54.476090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:54.560570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1424-cdcc-4e80-8435-d9353a545889 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.575117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.606427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.607372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.607758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.608170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.609673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.611342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.613407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 9/19] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:22:54.620429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:54.635616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4611-75fb-4e44-a352-1df4f257c6bd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.679050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.679787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.680132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.680462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.681552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:22:54.681923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.683457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f22d6e-1ce6-45bc-8c9f-0b612e5ffa8b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.704811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 11/20] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:22:54.759046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:54.801368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085e0b-e490-4b02-b643-688a9039a64b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:54.870026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:54.871250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:54.871693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:54.872189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:54.873936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.domain.id=default, user.id=cf4ea7b7962d4647b8de9301728f33d1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:22:54.874550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:54.877929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f63c-0a38-4c1e-b476-4d1f9b175ad3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:54.891926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 10/21] 10.222.0.77 () {56 vars in 1322 bytes} [Mon Jul 18 16:22:54 2022] GET /identity/v3/role_assignments?user.id=cf4ea7b7962d4647b8de9301728f33d1&scope.domain.id=default&role.id=54d519e3caa64c32a9930f925d38c156 => generated 233 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:22:58.098115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2d907-7d90-479d-a784-cb6ee201cb6b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:58.098115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2d907-7d90-479d-a784-cb6ee201cb6b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:58.098115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2d907-7d90-479d-a784-cb6ee201cb6b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:58.098115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 12/22] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:22:58.108477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fe0c-b556-4a41-9782-90bd17f4378c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:58.109785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fe0c-b556-4a41-9782-90bd17f4378c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:58.111132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fe0c-b556-4a41-9782-90bd17f4378c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:58.152009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3430fe0c-b556-4a41-9782-90bd17f4378c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:58.154603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3430fe0c-b556-4a41-9782-90bd17f4378c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:58.327206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 11/23] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:22:58.405946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2d907-7d90-479d-a784-cb6ee201cb6b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:58.436438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37636f58-db04-4484-aac9-e8b842393f08 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:58.437361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37636f58-db04-4484-aac9-e8b842393f08 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:58.438096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37636f58-db04-4484-aac9-e8b842393f08 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:58.445427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 12/24] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:58.448090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:58.448880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:58.449601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:58.492178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:58.494544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:58.692092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 13/25] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:58.725383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2d907-7d90-479d-a784-cb6ee201cb6b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:58.731434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:58.739025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:58.783260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:58.837872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:58.839041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:58.839535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:58.840027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:58.841519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:58.844232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05d76d-a183-41aa-9915-f71a7c1b560e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:59.022262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e9af-1b25-494a-9997-f90dfa294afc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:59.087098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:59.088910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.089657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.090472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.110970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:59.114769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:59.137422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 13/26] 10.222.0.77 () {60 vars in 1131 bytes} [Mon Jul 18 16:22:58 2022] POST /identity/v3/roles => generated 219 bytes in 726 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:22:59.137422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d29d-ce36-4f9a-9a5b-6a497aecdbe7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.137422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d29d-ce36-4f9a-9a5b-6a497aecdbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.137422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d29d-ce36-4f9a-9a5b-6a497aecdbe7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.138802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 14/27] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:59.140317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f19ea-1b95-4c70-b852-8eac49221f10 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.140818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f19ea-1b95-4c70-b852-8eac49221f10 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.141305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f19ea-1b95-4c70-b852-8eac49221f10 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.150778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 15/28] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:59.153451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8f7f-65f7-4a3c-8235-5057f1383324 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.153938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8f7f-65f7-4a3c-8235-5057f1383324 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.154442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8f7f-65f7-4a3c-8235-5057f1383324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2490f417-5cc2-4774-8282-20ae9d72cf0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:22:59.164288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:22:59.165760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:22:59.167876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:22:59.167876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:22:59.167876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:22:59.167876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:22:59.171114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 14/29] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:22:59.174346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.175236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.175945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.204475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:59.207129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:59.216239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6f1d8f7f-65f7-4a3c-8235-5057f1383324 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:59.218909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6f1d8f7f-65f7-4a3c-8235-5057f1383324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:59.356976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:59.385712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 15/30] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:59.386936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:59.387795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 16/31] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:59.389227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd855-6743-458c-b0f0-d74f82ae1de1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.389602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd855-6743-458c-b0f0-d74f82ae1de1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.389981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd855-6743-458c-b0f0-d74f82ae1de1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.391765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 17/32] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:59.397542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6334-eeea-4b64-93ac-8735c38e85da None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.397968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6334-eeea-4b64-93ac-8735c38e85da None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.398330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6334-eeea-4b64-93ac-8735c38e85da None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.404025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 18/33] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:22:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:22:59.404025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcca79-ff62-4f1a-afee-d4836313a1c3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.404025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcca79-ff62-4f1a-afee-d4836313a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.404025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcca79-ff62-4f1a-afee-d4836313a1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.426455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-befcca79-ff62-4f1a-afee-d4836313a1c3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:59.426455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-befcca79-ff62-4f1a-afee-d4836313a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:59.519698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 19/34] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:59.524835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.525323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.525803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.552012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:22:59.553948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:22:59.565766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbbcc-b5ac-46a6-b503-3a59cc2cc778 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:59.594900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:59.615412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:59.616171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.616539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.617600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.618598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:22:59.618995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:59.620527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:59.639742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 16/35] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:22:59.640611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:59.640796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 20/36] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:22:59.647531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:59.711788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:59.769546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b976aeb-06af-4bef-b80e-4dd7d51f2336 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:59.775219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6431061-0e1b-48c7-ab85-7d41a289b71d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:22:59.808978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:59.809966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.810248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.810566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.811355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:22:59.812947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:22:59.818560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:22:59.819510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:22:59.819764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:22:59.820070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:22:59.828862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 21/37] 10.222.0.77 () {60 vars in 1131 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/roles => generated 225 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:22:59.830275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:59.832617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:22:59.840947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:22:59.841927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:22:59.842780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 17/38] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:22:59.844134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:22:59.882182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:22:59.995905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb711-2125-4eb7-86cb-091e5165fa8a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.004174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8d83-5077-4c1a-a8ed-b45e8c87104d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.005092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:00.038090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.038862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.039157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.039441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.040182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.041712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.054993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.055708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.056190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.056361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.057094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.058626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.077111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 22/39] 10.222.0.77 () {60 vars in 1131 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:00.077985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:00.184674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:00.205486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4b89-f18c-49aa-8e37-8c81bc0432bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.230669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 18/40] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:22:59 2022] POST /identity/v3/projects => generated 301 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:00.234131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:00.252593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.253491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.253838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.254145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.257092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.259538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.262090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 23/41] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:00.268410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:00.394960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee28f2-62c8-4774-9816-1f2d600facbd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.412685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f2de8-93f0-4ad4-8ee7-7a5a7a2b40d9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.458562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.458821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.459184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.459508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.459646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.459781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.459895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.460010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.460494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:00.460739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.460882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.462212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.462345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.473747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 19/42] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:00.480901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:00.506820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 24/43] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:23:00 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:00.516477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:00.628999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684ff6-1608-489e-9921-bcfb16d3ecce None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.690984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.692681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.693342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.693862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.696672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.699593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.703091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 20/44] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:00 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:00.710734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:00.722029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:00.896545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f424c-ae79-4d26-893d-251d3e28af2a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:00.961606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:00.962917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:00.963420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:00.963916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:00.966186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:00.968994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16176db-f9f9-4aa2-ad46-78edced69435 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:00.972104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 25/45] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:00.978444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:01.201975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:01.218911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da681d7-4180-481f-865f-f4337ca786e3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:01.264444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:01.265388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:01.265615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:01.265892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] PATH_INFO: `/v3/domains/default/users/cf4ea7b7962d4647b8de9301728f33d1/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:01.268280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cf4ea7b7962d4647b8de9301728f33d1, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:01.269861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4292c7-f038-4d65-a0cc-f57917746cbf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:01.287491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 21/46] 10.222.0.77 () {58 vars in 1361 bytes} [Mon Jul 18 16:23:00 2022] PUT /identity/v3/domains/default/users/cf4ea7b7962d4647b8de9301728f33d1/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:23:02.717903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ab44-b8ef-4849-88a7-45258533755c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:02.718514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ab44-b8ef-4849-88a7-45258533755c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:02.718917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ab44-b8ef-4849-88a7-45258533755c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:02.721658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 26/47] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:02.730818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:02.732100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:02.732843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:02.753631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:02.755274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:02.849327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 22/48] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:02.872950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040ab44-b8ef-4849-88a7-45258533755c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:02.976089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:03.026060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040ab44-b8ef-4849-88a7-45258533755c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:03.084643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:03.085460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:03.085745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:03.086026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:03.087380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:03.088884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbefe78-021d-4c1b-adae-1a436de0b7c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:03.090648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 27/49] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:03.099514 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:03.243157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:03.287459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbddf5-35fa-4257-8900-217b838a086b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:03.352317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:03.353429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:03.353918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:03.354713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:03.356377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:03.359110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c16e7-f39a-42d5-a029-aebe3fdf3e85 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:03.406497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 23/50] 10.222.0.77 () {60 vars in 1140 bytes} [Mon Jul 18 16:23:03 2022] POST /identity/v3/projects => generated 298 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:23:03.421318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60c51-84cd-40ca-b808-14521771157f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:03.421631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60c51-84cd-40ca-b808-14521771157f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:03.421900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60c51-84cd-40ca-b808-14521771157f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:03.424411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 28/51] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:03.430892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:03.431214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:03.431472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:03.463145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:03.465332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:03.553971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 24/52] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:03.578323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab60c51-84cd-40ca-b808-14521771157f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:03.725396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab60c51-84cd-40ca-b808-14521771157f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:03.783698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:03.784542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:03.784812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:03.785088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:03.799905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:03.811577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:03.812558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:03.813997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:03.814627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 29/53] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:03.823217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:03.896236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:03.951406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a05a-3ee2-4d48-af45-2ad3a4fc1e46 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:04.000526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:04.001699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:04.002211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:04.002717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:04.004201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:04.004751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:04.007505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:04.023442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 25/54] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:04.029491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:04.033454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:04.152160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c70e-067d-48df-a5ad-44dd5379dfc9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:04.198184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:04.199713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:04.200223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:04.200734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:04.203528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:04.206885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:04.209810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 30/55] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:04 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:04.216023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:04.358969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575ceaf-a72e-41c5-914d-15d61e480d53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:04.389683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:04.406874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:04.408878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:04.409384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:04.409871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:04.412265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:04.415117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12eb118-710d-47ce-bda3-fa45591a02fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:04.418704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 26/56] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:04.424982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:04.512623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:04.552192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370906b3-56e6-4f4a-9440-b96f14e2573d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:04.600048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:04.601199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:04.601710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:04.602197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:04.603886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.domain.id=default, user.id=cf4ea7b7962d4647b8de9301728f33d1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:04.604458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:04.607341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680af3cd-7afb-406a-92ff-257a117ca7cf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:04.623780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 31/57] 10.222.0.77 () {56 vars in 1322 bytes} [Mon Jul 18 16:23:04 2022] GET /identity/v3/role_assignments?user.id=cf4ea7b7962d4647b8de9301728f33d1&scope.domain.id=default&role.id=54d519e3caa64c32a9930f925d38c156 => generated 530 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:05.685093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a11c6-1cbc-4c3c-85e0-35f5a77a8cb9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:05.685612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a11c6-1cbc-4c3c-85e0-35f5a77a8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:05.685766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a11c6-1cbc-4c3c-85e0-35f5a77a8cb9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:05.687314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 27/58] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:05.693427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:05.693930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:05.694056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:05.822864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:05.824251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:05.899830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 32/59] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:05.924789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a11c6-1cbc-4c3c-85e0-35f5a77a8cb9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:06.039840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a11c6-1cbc-4c3c-85e0-35f5a77a8cb9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:06.086229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:06.087081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:06.087366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:06.087640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:06.088930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:06.090435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f72ba-f80d-453f-adcf-b822aa4a7e62 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:06.092122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 28/60] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:06.098645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:06.193286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:06.235640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f70a7-f0de-4c0d-bdc0-cc48316c0746 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:06.274932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:06.275627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:06.275915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:06.276196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:06.276975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:06.278503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d57a7-7b19-489e-9d2f-211357183c87 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:06.308603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 33/61] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:06 2022] POST /identity/v3/users => generated 275 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:23:09.753838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd635-729e-4873-8612-cb904fe31729 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:09.754878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd635-729e-4873-8612-cb904fe31729 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:09.755407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd635-729e-4873-8612-cb904fe31729 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:09.757073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 29/62] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:09.763046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba52ff-51db-4a69-abc3-358a4d9a7d9d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:09.763994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba52ff-51db-4a69-abc3-358a4d9a7d9d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:09.764514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba52ff-51db-4a69-abc3-358a4d9a7d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:09.791532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cbba52ff-51db-4a69-abc3-358a4d9a7d9d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:09.793098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cbba52ff-51db-4a69-abc3-358a4d9a7d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:09.897025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 34/63] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:09.899506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f169d-c890-4ca7-91aa-13fca4780b1f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:09.900096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f169d-c890-4ca7-91aa-13fca4780b1f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:09.900570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f169d-c890-4ca7-91aa-13fca4780b1f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:09.902295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 30/64] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:09.908702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:09.909292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:09.909815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:09.920814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f169d-c890-4ca7-91aa-13fca4780b1f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:09.943236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:09.944487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:10.055411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.056721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.101022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 35/65] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:10.125333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.141327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f169d-c890-4ca7-91aa-13fca4780b1f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.221012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.221841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.222120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.222392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.241347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:10.250000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:10.252314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:10.254579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 31/66] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:10.258620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.352117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350e43d-3104-48d6-a7a8-e7d61f0d2f64 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.377660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76459eb-8694-4124-8010-29caccba0b62 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.392144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.392972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.393265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.393549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.398325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.412204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:10.420819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-03c8f0f8-d509-47b6-af2e-1756149ae14c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:10.421049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:10.421823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:10.422664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:10.423238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.423238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.423238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.423401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 36/67] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:10.423473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:10.424125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:10.424281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f298-7158-4d7c-8e2e-8d02108273fd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.424281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f298-7158-4d7c-8e2e-8d02108273fd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.424538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f298-7158-4d7c-8e2e-8d02108273fd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.426959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:10.427204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 37/68] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:10.433597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.433892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.434170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.441277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 32/69] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:10.442270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.463565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:10.464822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:10.505618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.554582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 38/70] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:10.555575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.639916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3dbd2-7013-4804-ba4e-b44fadf0e066 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.683055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.683742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.684024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.684297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.685063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:10.685369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:10.686839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7468d-5674-431e-861a-c615142bd8da None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:10.696461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 33/71] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:10.697837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c778-83f8-4831-8520-dbc354a19b86 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.698130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c778-83f8-4831-8520-dbc354a19b86 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.698434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c778-83f8-4831-8520-dbc354a19b86 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.700028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 34/72] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:10.701007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dd7a2-5f2e-4ed2-b15d-928803e70132 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.701343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dd7a2-5f2e-4ed2-b15d-928803e70132 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.701583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dd7a2-5f2e-4ed2-b15d-928803e70132 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.706474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 35/73] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:10.706474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dd7a2-5f2e-4ed2-b15d-928803e70132 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.706707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ac946-a7a4-446e-a8b1-8cd02e029542 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.754107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.755770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.756214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.756738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.767099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.786747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:10.789731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:10.798500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 39/74] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:10.798500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:10.835235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dd7a2-5f2e-4ed2-b15d-928803e70132 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:10.844023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:10.878949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:10.879886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.880255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.880614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.892999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e47e8168-33fd-44c6-a10f-c8fcaac289de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:10.901258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:10.902488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 36/75] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:10.903918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:10.905245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:10.905245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:10.922705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:10.923965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:10.958617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11f24f-82ef-47ee-baf7-6b1fb51f3399 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.002256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.003234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.003546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.003884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.005277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.007049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19558162-0974-4aa3-b9e0-fade9260c032 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.008879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 40/76] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:10 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:11.010187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.010523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.010863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.018965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 37/77] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:11.020053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.049382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:11.051327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:11.066270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.144026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 41/78] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:11.144921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.168361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51dcf7-ae07-4e29-acc9-9592507825ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.196445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.210059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.210768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.211058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.211339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.291043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673db6-3a65-4ab8-b05c-b104c4dc4976 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.320170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.323081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.331298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.332065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.332403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.332683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.333456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:11.333765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.335206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.349130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 42/79] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:11.350016 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.366778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.451458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 38/80] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:11.452335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.474824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.489608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956cbf0-bd1a-404a-aa08-30a51b37be83 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.536700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.537755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.538221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.538704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.623334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1b2b1-d0d5-498a-b426-cd486df8b5fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.623543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.625061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.666941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.672848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.673675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.673964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.674239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.675577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.677083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.679165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 39/81] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:11.679687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.752759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.776115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 43/82] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:11.778222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.805900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea7d6-bfaf-4abe-b1ad-0f16e9344f1a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.856967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.857795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.858083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.858394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.870604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:11.878690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:11.879623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:11.880603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 40/83] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:11.881501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:11.918288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61114-b48c-432e-8d4e-27f8a2cbe0d9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:11.954508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:11.962326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:11.963264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:11.963550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:11.963835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:11.965196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:11.966815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:11.978038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 44/84] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:11.978038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.022392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb293a-747c-4529-b598-41e0173437a4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.056298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.071265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.072342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.072698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.073045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.074734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.076498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.084167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 41/85] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:11 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:12.084167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.120772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c299-12fb-446f-a965-1dae04b1d0d7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.161938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.162588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.162880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.163163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.163900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.165426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.187198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 45/86] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:11 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:12.188129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.247757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bbdf-b97f-47ef-be00-b3bdaa48dc6e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.284570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.295398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.296357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.296644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.296927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.298350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.303036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.303036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 42/87] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:12.303036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.396481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff4e26-92fe-46ea-baa9-5f00c4f4e6bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.424997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.456220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315849-c2a2-4ce7-bf5f-789cebc0f786 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.472106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.472834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.473267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.473558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.474549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:12.474887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.476416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.489789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 46/88] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:12.490792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.532392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.533084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.533369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.533646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.625200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cbe0-90e4-48cd-a106-ddf81d60339b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.657631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.676776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.678311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.692844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.695244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.696657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.697681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.700442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=623c485832d44648b870a433f97830f1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:12.701712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.705871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.721410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 47/89] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=8569379203d74bee9bdcc46f1e112633 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:12.723718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.815390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 43/90] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:12.816397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:12.861803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.938966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:12.941796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232262d-93b7-41f3-80e0-0c80149e99cb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:12.985916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:12.986867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:12.987154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:12.987440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:12.988823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:12.990330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:12.993482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 44/91] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:12.993482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:13.114483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578823c-0919-436b-b21a-d85b1208283e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:13.171862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bf69a-7d2c-4928-a5f5-fe44a0908923 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:13.223654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:13.224510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.224796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.225079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:13.226561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:13.228114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:13.232887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 45/92] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:13.233822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:13.236670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:13.237409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.237704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.237986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:13.239015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=cf4ea7b7962d4647b8de9301728f33d1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:13.241363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:13.242931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:13.253639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 48/93] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:12 2022] GET /identity/v3/role_assignments?user.id=cf4ea7b7962d4647b8de9301728f33d1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:13.259125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:13.281287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:13.492732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae8217-7ce1-46d4-8526-a03c80297b66 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:13.515850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2787092-8324-4f37-a565-1843d26ada73 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:13.520701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:13.595063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:13.595745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.596028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.596302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:13.659666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:13.660829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.661343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.661848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:13.663566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=623c485832d44648b870a433f97830f1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:13.664148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:13.667381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:13.687668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 49/94] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:13 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:13.813964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:13.816842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:13.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:14.072240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 46/95] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 839 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:14.079177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:14.351432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706dfa9-31fe-44bc-afdd-cf16168b9733 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:14.447834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17436d50-8d05-428a-817a-061de45c299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:14.449423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436d50-8d05-428a-817a-061de45c299f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:14.449964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436d50-8d05-428a-817a-061de45c299f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:14.450507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436d50-8d05-428a-817a-061de45c299f None admin] PATH_INFO: `/v3/projects/8709402e6933401798182653c644fe82` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:14.453051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17436d50-8d05-428a-817a-061de45c299f None admin] RBAC: Authorizing `identity:get_project(project_id=8709402e6933401798182653c644fe82)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:14.456472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17436d50-8d05-428a-817a-061de45c299f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:14.460554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 50/96] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:14 2022] GET /identity/v3/projects/8709402e6933401798182653c644fe82 => generated 312 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:14.467412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:14.698222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb49b3-657c-40f7-bec2-305d28925844 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:14.744501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:14.752102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:14.752837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:14.753151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:14.753391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:14.754259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=8709402e6933401798182653c644fe82, user.id=623c485832d44648b870a433f97830f1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:14.754614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:14.756203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f895b7-8a95-4d24-8a45-d94c486f5e71 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:14.767749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 47/97] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:14 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=8709402e6933401798182653c644fe82&role.id=8569379203d74bee9bdcc46f1e112633 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:15.960547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01d50-ba30-4699-b382-3cd18941d8b1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:15.962089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01d50-ba30-4699-b382-3cd18941d8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:15.962920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01d50-ba30-4699-b382-3cd18941d8b1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:15.966187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 51/98] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:15.975515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:15.976999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:15.977709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.010152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:16.012628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:16.154746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 48/99] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:16.194859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01d50-ba30-4699-b382-3cd18941d8b1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:16.279533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:16.458508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01d50-ba30-4699-b382-3cd18941d8b1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:16.553392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:16.554711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.555229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.555753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.559924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:16.562871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f73483-8b06-41d3-bbda-61ca408b1a3f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:16.571259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 52/100] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:16.580618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:16.853429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55246a-5328-4e8d-8adb-41713e0954e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:16.915668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:16.934747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:16.936571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.937380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.938205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.939996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:16.943508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d38f2-0aaa-47b1-912e-304f855635fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:16.944499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b6a4-82bd-489b-be28-6b0fe114d5b6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.945003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b6a4-82bd-489b-be28-6b0fe114d5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.945494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b6a4-82bd-489b-be28-6b0fe114d5b6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.948027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 53/101] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:16.958101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.958720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.959219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.993105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:16.995412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:17.011619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 49/102] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:23:16 2022] POST /identity/v3/projects => generated 302 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:17.018460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6102b-80be-43eb-a1c8-e2d67dcf9263 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.019470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6102b-80be-43eb-a1c8-e2d67dcf9263 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.020187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6102b-80be-43eb-a1c8-e2d67dcf9263 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.025410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 50/103] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:17.042725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.042725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.042725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.088438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:17.095173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:17.168936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 54/104] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:17.211886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.350960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:17.426278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 51/105] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:17.439245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95505bdf-36b8-4d23-a532-f1f0491fcb93 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:17.501746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.545584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:17.548225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.548703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.549217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.573774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:17.591435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:17.593219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:17.600124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 55/106] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:17.610489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.811269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:17.833376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635059-87ea-4a25-8460-6bbb4a6c8e4f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:17.900764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:17.901854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.902346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.902878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.904271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:17.904807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:17.907457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:17.907921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb263e-5e8f-4e27-ad3b-f2c7efc30298 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:17.923956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 56/107] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:17.933450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.935737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:18.023724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:18.025309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.025907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.026535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.053835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:18.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:18.076214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:18.077700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 52/108] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 573 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:18.085391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:18.229203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450a5a5-e528-48ef-bf40-65e41dd82b51 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:18.354495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:18.356485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.357228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.357934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.364396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:18.367693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:18.378979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 57/109] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:18.388543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:18.408737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2252b5-e060-4501-9018-c4bae22d6c05 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:18.478944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:18.514246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:18.515417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.515919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.516967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.518368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:18.518943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:18.521637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25535d-c857-430f-ab6e-c642c7509a62 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:18.553658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 53/110] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:18.555864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95767c-c869-47ba-a654-338176cc7286 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.556343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95767c-c869-47ba-a654-338176cc7286 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.556830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95767c-c869-47ba-a654-338176cc7286 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.559413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 54/111] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:18.564629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95767c-c869-47ba-a654-338176cc7286 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:18.682962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:18.775081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853a23-f7c0-4c26-ae46-3de907a30557 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:18.900987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95767c-c869-47ba-a654-338176cc7286 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:18.903671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:18.904790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.905267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.905748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.907897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:18.910939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0408-045e-4608-92ef-f5aee1726c99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:18.914694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 58/112] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:18 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:18.917056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:18.917526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:18.917974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:18.950479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:18.951629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:19.054160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.055721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.056301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.056782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] PATH_INFO: `/v3/users/203e499782354c2194d0719ad3e993b6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.093054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 59/113] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:19.094666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.119775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] RBAC: Authorizing `identity:get_user(user_id=203e499782354c2194d0719ad3e993b6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.122621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.127885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 55/114] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:18 2022] GET /identity/v3/users/203e499782354c2194d0719ad3e993b6 => generated 289 bytes in 563 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:19.146464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.148112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.307089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ff9d7-f373-4192-a360-ab04aaa6426d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.354831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.372216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.373661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.374143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.374663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/623c485832d44648b870a433f97830f1/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.378541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user_id=623c485832d44648b870a433f97830f1, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.381236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.381901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb6a91-62fc-41e8-80d7-93011ebd936d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.410674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 60/115] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:19 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/623c485832d44648b870a433f97830f1/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:19.412380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.467708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.468803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.469352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.469843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.596182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.651304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499d4eb-e48b-4981-b2e2-202facaedc85 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.700336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.703306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7b865-3ace-4c33-91d2-842c777bc4cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.722962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.724758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.725446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.726186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.748102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ca913f85-1ce5-46b3-a939-b451557b0161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:19.757019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:19.757940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:19.758743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:19.761371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 61/116] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:19.763172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9f0c-ac3f-458e-97ee-e6246260dc56 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.763698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9f0c-ac3f-458e-97ee-e6246260dc56 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.764287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9f0c-ac3f-458e-97ee-e6246260dc56 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.765956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 62/117] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:19.774584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f9f0c-ac3f-458e-97ee-e6246260dc56 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.919849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f9f0c-ac3f-458e-97ee-e6246260dc56 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.933885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.966438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 56/118] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:19.966438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.966715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.967113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.976124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189619-a585-461f-9360-fa8936873804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.977196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189619-a585-461f-9360-fa8936873804 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.978000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189619-a585-461f-9360-fa8936873804 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.978592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189619-a585-461f-9360-fa8936873804 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.980725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56189619-a585-461f-9360-fa8936873804 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:19.981320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56189619-a585-461f-9360-fa8936873804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.983027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56189619-a585-461f-9360-fa8936873804 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.994830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 63/119] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:19.996042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189619-a585-461f-9360-fa8936873804 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.001306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:20.003059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:20.010542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.131653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 57/120] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:20.133819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.162438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189619-a585-461f-9360-fa8936873804 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.244180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.246503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.247951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.249173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.252024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.256578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbee9c7-cb2e-417e-bad9-aa1231c52951 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.267880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 64/121] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:20.267880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053d02e-9eb0-4914-ab55-fd3ed4027c18 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.267880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053d02e-9eb0-4914-ab55-fd3ed4027c18 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.267880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053d02e-9eb0-4914-ab55-fd3ed4027c18 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.274731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 65/122] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:20.276758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053d02e-9eb0-4914-ab55-fd3ed4027c18 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.336680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f449f5-2c66-45b7-9795-4906480eedfe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.377515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.408268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.420201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.422994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.423937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.424843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.427574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.430501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.434858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 58/123] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:20.436983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.445739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053d02e-9eb0-4914-ab55-fd3ed4027c18 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.487037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.488735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.489207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.489695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.506305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f6dc3531-bb2f-4e68-87ce-e728c6eb1e4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:20.516436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:20.517906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:20.519142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 66/124] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:20.520179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.520179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.520179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.546513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:20.548513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:20.657520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d73b-5b7d-4215-aad5-2e65c330a123 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.687028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 67/125] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:20.689777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.699313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.699982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.700263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.700538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.701390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=203e499782354c2194d0719ad3e993b6) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:20.701705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.703323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.710474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 59/126] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/role_assignments?user.id=203e499782354c2194d0719ad3e993b6&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=c85d184365d949ca99a130b6fff5392f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:20.714448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.786557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.842624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cea4ab-16ba-438f-8416-febb74c0dc45 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.882389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.883685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.884787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.885519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.885965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.888171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:20.888715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.890605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.900404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 60/127] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:20.906121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.029976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71adc4-9c8f-48ea-895d-0fb4dc8feb40 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.030717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28222e91-c2d4-482b-bceb-2d1ec5fe8b99 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.064634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:21.074897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41067-d178-4969-8926-148377e91285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.076079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41067-d178-4969-8926-148377e91285 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.076756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41067-d178-4969-8926-148377e91285 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.077211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41067-d178-4969-8926-148377e91285 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.093139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41067-d178-4969-8926-148377e91285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4fa41067-d178-4969-8926-148377e91285 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:21.102729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:21.103653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:21.104426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:21.107432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 61/128] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:21.109546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41067-d178-4969-8926-148377e91285 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.124846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.126504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.126966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.127448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.129737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.132383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.135535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 68/129] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:20 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:21.136956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.259498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41067-d178-4969-8926-148377e91285 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.302364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.303904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.304415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.304927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.306584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.308533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.310815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 62/130] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:21 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:21.311849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.322129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4748ff9-a4d7-495b-a43d-62aa6db8d3a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.365506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:21.368825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:21.381614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.382988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.383514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.383989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.385593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:21.386229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.388284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438dd49-d2d6-44fe-99b6-45e502a18972 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.402825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 69/131] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:21.408577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01452-cdca-44b7-9cd5-38285376375e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.409615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01452-cdca-44b7-9cd5-38285376375e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.410090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea01452-cdca-44b7-9cd5-38285376375e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.411995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 70/132] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:21.413130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea01452-cdca-44b7-9cd5-38285376375e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.474112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea729dc-a300-4304-89e4-eef73ed98cf9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.542087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.543734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.544252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.544748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/cf4ea7b7962d4647b8de9301728f33d1/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.545915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea01452-cdca-44b7-9cd5-38285376375e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.548681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user_id=cf4ea7b7962d4647b8de9301728f33d1, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.551684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.580803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 63/133] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:21 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/cf4ea7b7962d4647b8de9301728f33d1/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:21.582636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.635979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.637041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.637447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.637830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.646502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.646502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66308f3d-bff3-425a-af0b-479526cb3ee1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.647001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 71/134] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:21 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:21.648404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.648795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.649174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.708772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:21.711567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:21.793697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e6d2-cd59-428d-8ba6-36e03d248df0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:21.795791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:21.839433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:21.840236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:21.840521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:21.840796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:21.842226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:21.843759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:21.845761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 64/135] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:21 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:21.851200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:21.857616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:21.892875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 72/136] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:21.899331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.069322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d4cc8-ee97-4b60-a912-c1787fcaf468 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.083310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac689cb8-525e-45a1-91ca-b123cc55703d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.146062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.146905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.147188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.147452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/623c485832d44648b870a433f97830f1/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.150068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user_id=623c485832d44648b870a433f97830f1, role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:22.151581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:22.159277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.160764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.161283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.161796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] PATH_INFO: `/v3/projects/8709402e6933401798182653c644fe82` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.164323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=8709402e6933401798182653c644fe82)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:22.167516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:22.171407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 73/137] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:21 2022] GET /identity/v3/projects/8709402e6933401798182653c644fe82 => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:22.172842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.185545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 65/138] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:21 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/623c485832d44648b870a433f97830f1/roles/345ac5f37e244b4faa681fbd95404a3d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:22.191495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.246835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:22.385252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08bd9-0e63-4eaa-853f-28cb84782917 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.413617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:22.427984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.429047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.429339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.429613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] PATH_INFO: `/v3/projects/8709402e6933401798182653c644fe82/users/623c485832d44648b870a433f97830f1/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.432142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] RBAC: Authorizing `identity:create_grant(project_id=8709402e6933401798182653c644fe82, user_id=623c485832d44648b870a433f97830f1, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:22.433748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:22.454108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 66/139] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:22 2022] PUT /identity/v3/projects/8709402e6933401798182653c644fe82/users/623c485832d44648b870a433f97830f1/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:22.504743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236871d-2c58-4601-9670-2dcc95999fd9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.562649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.563610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.563897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.564180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.565648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:22.567238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f871e-1f2a-4b69-8ce6-7959ec3837d2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:22.569464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 74/140] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:22.575628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.698901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:22.723832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c97f1-8d4a-4cb4-88c0-b00b31328d89 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.799339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.800079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.800369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.800643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.801434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:22.803048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb955059-ba21-4524-b445-39c2b2928dbf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:22.843195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 67/141] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:22 2022] POST /identity/v3/users => generated 283 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:23.575794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b47c0-da9d-4b8c-af40-fa91b49e64ab None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.576329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b47c0-da9d-4b8c-af40-fa91b49e64ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.576329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b47c0-da9d-4b8c-af40-fa91b49e64ab None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.578162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 75/142] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:23.592319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb3b31-ac98-4df5-8dad-48d52011c093 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.593266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb3b31-ac98-4df5-8dad-48d52011c093 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.593740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb3b31-ac98-4df5-8dad-48d52011c093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.621824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f0bb3b31-ac98-4df5-8dad-48d52011c093 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:23.623444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f0bb3b31-ac98-4df5-8dad-48d52011c093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:23.728299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 68/143] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:23.770697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b47c0-da9d-4b8c-af40-fa91b49e64ab None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.940220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a84f43-c6b4-480c-b861-2d77752c80d9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.940674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a84f43-c6b4-480c-b861-2d77752c80d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.941150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a84f43-c6b4-480c-b861-2d77752c80d9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.951100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 69/144] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:23 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:23.965572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.966609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.969678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.990941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b47c0-da9d-4b8c-af40-fa91b49e64ab None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.023729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:24.028199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:24.079146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.101371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.102199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.102543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.102837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.137076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:24.160843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:24.162523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:24.163974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:24.166160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 76/145] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:24.171083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.237797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 70/146] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:24.260611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.281599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.384673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab750b-bfc8-41ec-8ff3-cdd1cd2dde56 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.429166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.435114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.435620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.436129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.437581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:24.438256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.441145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.457449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 77/147] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:24.472738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.534940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.583000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85291137-a27c-4c51-ac15-f982c401fa64 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.606179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f73f8-84e1-47ca-825d-1f6853732bb3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.648404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.649700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 78/148] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:24.664586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.696789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.697750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.697985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.698235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.738443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:24.750949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:24.752015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:24.752835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:24.755030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 71/149] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:24.761248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.795950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5565-e947-4a1c-8b21-5ccd3dd04e8c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.814381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.843779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.845021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.845465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.845893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.933570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.935607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.002807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:25.066126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 79/150] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:25.072376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:25.090354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:25.117899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e354a-3f84-44ce-8e73-922e48860034 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:25.198259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80292c20-90c7-4b69-87b2-ac815292eda7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:25.213251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:25.214787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:25.215498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:25.216181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:25.217835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:25.218628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:25.222135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.243527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:25.244736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:25.245222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:25.245644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:25.250444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 72/151] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:25.250716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:25.250716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.251500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 80/152] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:25 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:25.253317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:25.259587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:25.385767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:25.410056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3912792-82f0-49d6-91b9-d489af0e1107 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:25.450117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:25.451353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:25.451834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:25.452332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:25.453476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=623c485832d44648b870a433f97830f1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:25.454566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:25.456591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.463861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 81/153] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:25 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=8569379203d74bee9bdcc46f1e112633 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:25.553569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eb549-5f25-4d79-a1a6-2b0f7ddabab1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:25.629446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:25.661472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:25.663135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:25.663626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:25.664118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] PATH_INFO: `/v3/users/203e499782354c2194d0719ad3e993b6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:25.666521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=203e499782354c2194d0719ad3e993b6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:25.669418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.673456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 73/154] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:25 2022] GET /identity/v3/users/203e499782354c2194d0719ad3e993b6 => generated 289 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:25.679404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:25.888956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aec7a-01a1-448f-a2a4-d49d94aadb39 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:25.938533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:25.939456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:25.939765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:25.940063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:25.947489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:25.949003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b99d9-7e6e-4d70-ba67-e112943ad732 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:25.950976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 82/155] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:25 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:25.958464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:26.181717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240564-bdfc-4680-a478-51c87d5fa1d2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:26.227985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:26.229406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.229914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.230381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/203e499782354c2194d0719ad3e993b6/roles/c85d184365d949ca99a130b6fff5392f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.237251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user_id=203e499782354c2194d0719ad3e993b6, role_id=c85d184365d949ca99a130b6fff5392f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:26.240410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b49738-6e95-45c5-9de5-352c8614ce4d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:26.276118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 74/156] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:25 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/users/203e499782354c2194d0719ad3e993b6/roles/c85d184365d949ca99a130b6fff5392f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:26.462540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777f909-7c80-42ed-ae16-d90c4cfae2d1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.462953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777f909-7c80-42ed-ae16-d90c4cfae2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.463132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777f909-7c80-42ed-ae16-d90c4cfae2d1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.464935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 83/157] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:26.492469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.492892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.493381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.536446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:26.538519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:26.655767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:26.702596 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 75/158] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:26.747035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777f909-7c80-42ed-ae16-d90c4cfae2d1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:26.770631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:26.888129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777f909-7c80-42ed-ae16-d90c4cfae2d1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:26.933426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:26.934268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.934576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.934859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.948876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b55d4a69-3f52-44f0-94c0-febedb2a07e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:26.961972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:26.963735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:26.965247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 84/159] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:26.972158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.100336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fddd-3a13-48e4-8481-a6ae59411053 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.100336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fddd-3a13-48e4-8481-a6ae59411053 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.100710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fddd-3a13-48e4-8481-a6ae59411053 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.102084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 85/160] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:27.120395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.120799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.120948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.188374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:27.190435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:27.194643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782cb5-f5e6-4586-bfcb-d48a33d3d1af None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.271648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.273243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.273963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.274702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.276606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:27.277461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.280565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.311182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 76/161] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:27.317530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.425689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.461006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 86/162] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:27.508506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.517313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.546073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488df9a-9425-4d45-81d3-744d6ab9d5d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.622963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.625063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.626432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.627156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.632032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.640915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.644699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 77/163] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:27.652893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.916901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73ecc7-3971-4d94-a9ee-9aa2c2803ee8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.988812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb83d-9c98-4f6f-b131-2c4fb9805f29 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:28.014097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:28.014975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.015256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.015534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.024047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:28.048291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5a13cbf8-c74e-4bc5-9041-e549622c2134 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:28.063957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:28.065666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 87/164] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 558 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:28.067211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f593f-d9b6-4d48-8e1b-b959f9fa5229 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.068875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f593f-d9b6-4d48-8e1b-b959f9fa5229 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.068875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f593f-d9b6-4d48-8e1b-b959f9fa5229 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.073699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 88/165] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:28.075348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.075645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.075876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.106978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:28.108217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:28.123990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:28.124669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.124956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.125314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.243992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:28.268394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 89/166] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:28.269170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:28.367811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:28.369426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:28.553958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85203ae5-ea3c-4cd4-b191-ae88c4466415 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:28.607737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 78/167] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 957 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:28.608708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:28.635969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:28.676661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:28.678105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.678860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.679621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.686332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:28.687312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:28.690331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:28.722451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 90/168] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:28.724668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:28.800856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38edd1-e44f-4e2a-964d-af49a0c01d37 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:28.871169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:28.883322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:28.884077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:28.884364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:28.884646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:28.928075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:28.948597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:28.950350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:28.951914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:28.961806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 79/169] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:28.963674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:28.970433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b06678-870f-41ea-bccc-c140c3d75493 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.043790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.045657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.046179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.046707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] PATH_INFO: `/v3/projects/8709402e6933401798182653c644fe82` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.051437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] RBAC: Authorizing `identity:get_project(project_id=8709402e6933401798182653c644fe82)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:29.055611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58d9ae-631a-4fbd-a5df-6c1e51b4b3df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:29.059929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 91/170] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:28 2022] GET /identity/v3/projects/8709402e6933401798182653c644fe82 => generated 312 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:29.063266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1df7d-14af-43c6-8d8a-49b5e56914b8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.064124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1df7d-14af-43c6-8d8a-49b5e56914b8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.064908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1df7d-14af-43c6-8d8a-49b5e56914b8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.067912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 92/171] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:29.069716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1df7d-14af-43c6-8d8a-49b5e56914b8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:29.218217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.261641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07314302-3828-4765-a1e1-ed31632a3976 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.303541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1df7d-14af-43c6-8d8a-49b5e56914b8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.320823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.322284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.322800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.323270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] PATH_INFO: `/v3/users/cf4ea7b7962d4647b8de9301728f33d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.325364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4ea7b7962d4647b8de9301728f33d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:29.327986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:29.331989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 80/172] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:28 2022] GET /identity/v3/users/cf4ea7b7962d4647b8de9301728f33d1 => generated 247 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:29.333333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:29.351461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.372072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.373336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.373798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.374254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.376548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:29.377055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:29.379668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb04b-cb2e-4221-8bcb-e3292e00d0c6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:29.399255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 93/173] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:29.399255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ac74-6d77-4b16-b2d6-2fe11b47a6ca None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.399255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ac74-6d77-4b16-b2d6-2fe11b47a6ca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.399255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ac74-6d77-4b16-b2d6-2fe11b47a6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.424953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fd76ac74-6d77-4b16-b2d6-2fe11b47a6ca None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:29.426957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fd76ac74-6d77-4b16-b2d6-2fe11b47a6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:29.505757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7db7-bbc4-4dd3-a315-34ea3cb0c7d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.546444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 94/174] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:29.546444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d6cd8-1c7e-492b-8654-ff08ae23f41b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.546444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d6cd8-1c7e-492b-8654-ff08ae23f41b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.546444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d6cd8-1c7e-492b-8654-ff08ae23f41b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.547494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 95/175] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:29.549574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e08b-e29d-4395-957a-4096377b9223 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.550082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e08b-e29d-4395-957a-4096377b9223 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.550614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e08b-e29d-4395-957a-4096377b9223 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.554268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 96/176] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:29.556408 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e08b-e29d-4395-957a-4096377b9223 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:29.564020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.564708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.564992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.565271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.566182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=8709402e6933401798182653c644fe82, user.id=623c485832d44648b870a433f97830f1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:29.566552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:29.568075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:29.575394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 81/177] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:29 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=8709402e6933401798182653c644fe82&role.id=8569379203d74bee9bdcc46f1e112633 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:29.576260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:29.597306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.730087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0122-d8c0-4dfb-82d0-5335e11392c2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.805206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.806047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e08b-e29d-4395-957a-4096377b9223 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:29.807107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.807800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.808510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] PATH_INFO: `/v3/users/623c485832d44648b870a433f97830f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.811562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] RBAC: Authorizing `identity:get_user(user_id=623c485832d44648b870a433f97830f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:29.815415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e13932-a230-44ce-8e58-9ab373d00208 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:29.819089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 82/178] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:29 2022] GET /identity/v3/users/623c485832d44648b870a433f97830f1 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:29.821753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4722a5-7cf4-427b-bbdf-5aa9d96d4683 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.824140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4722a5-7cf4-427b-bbdf-5aa9d96d4683 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.824140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4722a5-7cf4-427b-bbdf-5aa9d96d4683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.851468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.854911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8f4722a5-7cf4-427b-bbdf-5aa9d96d4683 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:29.857545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8f4722a5-7cf4-427b-bbdf-5aa9d96d4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:29.881435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:29.882121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.882389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.882902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.963843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 83/179] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:29.966031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.966464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.966922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.977181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.996936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:29.998914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:30.032775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.035220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.130860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 84/180] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:30.130860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.192711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 97/181] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:30.194200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.218319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.253143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954d0e7-c1ce-4883-9f2e-e42df5712423 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.294645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.295769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.296147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.297477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.299156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.300767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.302529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 85/182] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:30.303344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.306464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.357168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718715-f3eb-4e16-b5a2-f2ce09ad38cc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.423433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.424535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.425005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.425468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.434033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a5e6-6b31-4356-adaf-c1ebf87c9492 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.477815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.478816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.479160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.479504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.492755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e5d3d86a-133f-4d4e-8ef5-2f48ef67e9d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:30.502189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:30.504058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:30.505602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 86/183] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:30.508466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed497-5f38-4e8b-88ac-458dc5709b72 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.508946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed497-5f38-4e8b-88ac-458dc5709b72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.509345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ed497-5f38-4e8b-88ac-458dc5709b72 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.514609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 87/184] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:30.516153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed497-5f38-4e8b-88ac-458dc5709b72 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.525171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.528018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.557313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.597062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.661196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 98/185] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:30.662213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.668133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ed497-5f38-4e8b-88ac-458dc5709b72 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.721385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.722300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.722697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.723042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.739574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:30.748867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:30.751192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:30.752731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 88/186] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:30.753945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.823287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74cfc-118e-4993-a297-3a99dc31ee04 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.845151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.871181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.872601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.873417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.873836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.876042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.878735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.881838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 99/187] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:30.883023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.903412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1f22-75d5-4855-97b1-7eca30666a3f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.930688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:30.944804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.945573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.945992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.946387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.947238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.948914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a09f1-c958-477e-a1fd-7c04f2a20694 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.970798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 89/188] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:30 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:30.972294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.972497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.972776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.993479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:30.994865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:31.060795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bf635-64b4-4728-836a-96a3dcd243d3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.106928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.107629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.107912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.108192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.109025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:31.109336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.111390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.121134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 90/189] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:31.123464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.127237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 100/190] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.132189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.229288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:31.324954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368ab7-5092-4985-b5a2-b8f8bc5daf07 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.336759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9985b59-26e6-41ad-bf51-43c4e13743bc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.347868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:31.396672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.397753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.398039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.398313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.401151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.402719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.405007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 91/191] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.405692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.407899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.409036 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.409546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.410021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.411507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:31.412356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.414938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.430972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 101/192] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.432214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.543327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380927-c210-4fb4-9c3e-1671882e351c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.588348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.589438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.589919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.590592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.594775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=cf4ea7b7962d4647b8de9301728f33d1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:31.595414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.597165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.604756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 92/193] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/role_assignments?user.id=cf4ea7b7962d4647b8de9301728f33d1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.606052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.637488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5b05-1f7b-4b93-9568-1221f68b5fdd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.661545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:31.673083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:31.712327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.713835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.714339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.714865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.739178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.740735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.742834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 102/194] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.743568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.749650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3548b4-3c01-4421-aaf3-e860df707954 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.793476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.794806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.795295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.795865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.809141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:31.818249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:31.819223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:31.820008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:31.823087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 93/195] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:31.829433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.866487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b732f-8f5b-43b6-bd4f-5b52b2cb0fc5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.908281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:31.909046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:31.909295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:31.909605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:31.910543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=623c485832d44648b870a433f97830f1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:31.910872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:31.912448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:31.920132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 103/196] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/role_assignments?user.id=623c485832d44648b870a433f97830f1&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:31.922470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:31.963874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d239-b576-46ff-98d3-b30f82d1f0ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:31.967853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.017356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.034001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.035077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.035407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.035642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.037170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.038757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.041152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 94/197] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:32.044458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:32.156866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f96d3-9057-43f7-9ebe-3b528f3b9e86 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:32.217453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.218350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.218679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.219648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.272755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec08a6-f110-4d70-8d38-4b9266dc9cd1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:32.315491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.316101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.316379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.316656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.317441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:32.317750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.319240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.329414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 95/198] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:32.330300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:32.366140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.393470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.396622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.450168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79008e1e-4b1e-47f1-8e05-27f562f79f00 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:32.460642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.492917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.493584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.493866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.494144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.581909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.583506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.617983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 104/199] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:32.619584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.621221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:32.711140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 96/200] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:32.711808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:32.835740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be3aac-56fc-4e61-9c0d-26063f5236e4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:32.841103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d015b-1898-40b3-89a7-4bf84b6c013b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:32.868168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.875555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.876597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.876945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.877285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.878782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.880347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.882641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 97/201] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:32.888535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:32.904352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:32.917330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:32.919323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:32.920055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:32.920921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] PATH_INFO: `/v3/users/203e499782354c2194d0719ad3e993b6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:32.923807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] RBAC: Authorizing `identity:get_user(user_id=203e499782354c2194d0719ad3e993b6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:32.927173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:32.931029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 105/202] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/users/203e499782354c2194d0719ad3e993b6 => generated 289 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:32.932764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.008305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0943b-b103-4883-9caa-a8cb778a8c83 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.063975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.065182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.065716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.066138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.067717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:33.069384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:33.071602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 98/203] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:33.072600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.145818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c2b8d-a33c-47ce-ae93-51867ed8e31c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.172998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.191007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.192329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.192798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.193252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.194347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=0eec17e4b52b4bb8bdf212b568c50f07) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:33.194895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:33.196873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:33.203732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 106/204] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:32 2022] GET /identity/v3/role_assignments?user.id=0eec17e4b52b4bb8bdf212b568c50f07&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:33.205047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.243778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf4b4-b348-4527-9bc8-523ee0eab621 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.299816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.300892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.301465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.301943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.357422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.463516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb754e9f-1851-4543-9756-72987cb314ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.489047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:33.491202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:33.512418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.513388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.514588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.515031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.516373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=0eec17e4b52b4bb8bdf212b568c50f07) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:33.517037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:33.520752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:33.535538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 107/205] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/role_assignments?user.id=0eec17e4b52b4bb8bdf212b568c50f07&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:33.611329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.689136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 99/206] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:33.699058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.846035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c58ec-dced-46a1-afb2-a99fd951a60e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.899645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.917214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.919374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.920127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.920844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.924056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:33.927319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85231a-1b6a-48c3-952f-270de216e2ee None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:33.931023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 108/207] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:33.940921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.155843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108441-89f8-45f5-b576-061b7d81cc65 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.224647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.226082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.226821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.227516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.229398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4, user.id=203e499782354c2194d0719ad3e993b6) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:34.230169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.233243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037238e8-69e7-4fe3-a23c-e832babd339c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.246167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 100/208] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/role_assignments?user.id=203e499782354c2194d0719ad3e993b6&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=c85d184365d949ca99a130b6fff5392f => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:34.371916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ab3d-3921-4aac-a585-d45d4f7354f7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.373353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ab3d-3921-4aac-a585-d45d4f7354f7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.374069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ab3d-3921-4aac-a585-d45d4f7354f7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.377645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 109/209] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:34.384988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.385788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.386549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.428397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:34.430923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:34.508872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.611393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 101/210] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:34.651904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075ab3d-3921-4aac-a585-d45d4f7354f7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.884222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075ab3d-3921-4aac-a585-d45d4f7354f7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.962271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.963137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.963422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.963693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.980304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-949559ee-e79e-418c-a9dc-56a1933c987c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:34.988622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:34.990262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:34.991855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 110/211] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:34.999422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.049623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1b52-c2fa-4fc2-9c1d-d9212895ffeb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.050206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1b52-c2fa-4fc2-9c1d-d9212895ffeb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.050691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1b52-c2fa-4fc2-9c1d-d9212895ffeb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.053254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 111/212] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:35.062116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.063551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.064297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.096108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:35.099561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:35.182793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb895f-4168-4067-8017-aad2ff842f8f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.220445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.234241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.235364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.235794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.236241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.237498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:35.237960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.239811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.257850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 102/213] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.259199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.275742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 112/214] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:35.311903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.316270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.467582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9eb8-436a-4881-b27b-41c3d6ff2831 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.519801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5cf1c-a8fe-4353-854d-97477458bd82 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.539648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.542151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.542911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.543580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] PATH_INFO: `/v3/users/8a372fb582674bb896fa44dce4b1b192` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.572356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] RBAC: Authorizing `identity:get_user(user_id=8a372fb582674bb896fa44dce4b1b192)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.575344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.579979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 103/215] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/users/8a372fb582674bb896fa44dce4b1b192 => generated 297 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.590645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.608592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.615784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.616667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.617378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.621419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.625277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.630305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 113/216] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:35.640518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.751904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.776145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.819484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd754-2a50-4a3d-a70d-5bcbfbc229b8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.846643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75df8a-b79c-45b5-bc7f-b7491e181123 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.896746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.897933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.898453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.899186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.920709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.923305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.924132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.924892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.926400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.929617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681fc83-24fa-4af5-8158-0015374479c2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.978073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 114/217] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:35 2022] POST /identity/v3/users => generated 297 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:36.057329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd331-4b98-470f-9fd5-a297bfafdda4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.057551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd331-4b98-470f-9fd5-a297bfafdda4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.057790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd331-4b98-470f-9fd5-a297bfafdda4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.059787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 115/218] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:36.060431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.063904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.066245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.066572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.066855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.092980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:36.095745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:36.196155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.234322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 104/219] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:36.240759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.244185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 116/220] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:36.283242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.369881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.424470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5ab9-38df-4cfa-be25-3bd53e743b4a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.523945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.526356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.527314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.528081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.529662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204256ac-20a8-4c38-8f0c-e150eb0e4bdc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.532121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.536769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.541482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 105/221] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:36.551463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.615134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.617153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.618198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.618983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.646126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:36.662114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:36.663941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:36.665515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:36.667620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 117/222] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:36.676009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.687015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.832800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae57d6-ac4d-4c67-ae11-0421e6027fc7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.917594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.918750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.919256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.919745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.921661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=8a372fb582674bb896fa44dce4b1b192) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:36.922264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.925212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b8d34-87da-4b47-801c-043f3beca032 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.938681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 106/223] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/role_assignments?user.id=8a372fb582674bb896fa44dce4b1b192&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=8569379203d74bee9bdcc46f1e112633 => generated 259 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:36.962090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.967973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a57c0-7b4a-4b64-92a0-ef533e669767 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.968563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a57c0-7b4a-4b64-92a0-ef533e669767 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.969667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a57c0-7b4a-4b64-92a0-ef533e669767 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.972906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 107/224] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:36.973115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0494a68-2cfa-4c8f-8ef6-17db88d65331 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.984128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.985135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.985640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:37.031968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:37.033769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:37.034526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:37.034816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:37.035098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:37.035909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:37.036222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:37.036824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:37.037677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:37.073993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 118/225] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:37.084003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:37.238074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 108/226] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:37.262599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:37.329583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770c6a-0ea7-4219-a31c-15ea1d051c32 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:37.375091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:37.419830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:37.422794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:37.423650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:37.424405 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:37.429108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:37.432920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:37.445699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 119/227] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:37 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:37.454605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:37.502250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e2ddb-e813-45ad-9f89-0cbc5918654c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:37.551690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:37.627679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:37.629583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:37.630061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:37.630622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:37.653197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fd4ac-bf19-400d-aced-6e9f0b8bd2e0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:37.676487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:37.692211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:37.694018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:37.695548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 109/228] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:37.706924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:37.708071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:37.709447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:37.710265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:37.711840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:37.717721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:37.722373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:37.726383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 120/229] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:37 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:37.745020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:37.962341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57939bc0-7313-45ec-a12d-d8031b0793e2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:37.966076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:37.994089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:38.018380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:38.019171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cfca-7837-47f7-be19-24fe9ce0982c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:38.019711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.020206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.020690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.021723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:38.022604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:38.024394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:38.036971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 110/230] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:38.043222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:38.117456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:38.118849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.119355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.119876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/0eec17e4b52b4bb8bdf212b568c50f07/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.122842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, user_id=0eec17e4b52b4bb8bdf212b568c50f07, role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:38.125418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:38.145226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 121/231] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:37 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/0eec17e4b52b4bb8bdf212b568c50f07/roles/345ac5f37e244b4faa681fbd95404a3d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:38.302482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc952aa-3fe5-4952-a589-7545bde60ce6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:38.373960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:38.375418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.375941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.376426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.379029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:38.380967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:38.383155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 111/232] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:38 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:38.389860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:38.652786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41b429-56c9-4599-b432-ca99f0235dc0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:38.745746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:38.747347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.747868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.748377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.751356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:38.754255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5dd2d-c782-4e42-ba6d-1295a4f67241 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:38.762459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 122/233] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:38 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:38.769333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:38.828693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.023375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1d44-fe74-4d3c-85ed-3e4fa15f5766 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.096012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.097680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.098191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.098782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/0eec17e4b52b4bb8bdf212b568c50f07/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.102544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, user_id=0eec17e4b52b4bb8bdf212b568c50f07, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:39.105350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82af72-f564-4081-8685-248bcfb61997 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:39.133104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 112/234] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:38 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/0eec17e4b52b4bb8bdf212b568c50f07/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:40.255649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf193c-c2c6-4b4e-b2bb-211b9c664729 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.256722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf193c-c2c6-4b4e-b2bb-211b9c664729 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.257217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf193c-c2c6-4b4e-b2bb-211b9c664729 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.258979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 123/235] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:40.262067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cbfc9-ea92-41e9-b043-e104366ecfab None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.263040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cbfc9-ea92-41e9-b043-e104366ecfab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.263574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cbfc9-ea92-41e9-b043-e104366ecfab None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.265378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 113/236] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:40.271121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.271755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.272233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.284023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.284678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.285155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.306991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:40.308860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:40.315810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:40.317343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:40.357139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.444340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 124/237] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:40.470332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.521993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 114/238] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:40.563629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.616898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774d7aa-31fb-4ce5-90ed-8b9cd3b98cb0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.669173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.670349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.670850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.671301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.673222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.674992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.675737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.678802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 125/239] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:40.686093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.739488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.877588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018df9a-7e9f-414f-ab53-eec3797ec53c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.959077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.960471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.960980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.961461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.985791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:41.003803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:41.004599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:41.010499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 115/240] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:41.012720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.039368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944b640-8488-40a7-b315-e4a5e226b174 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.117341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.119283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.120033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.120730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.123859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.127882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f738-038e-48a6-9a78-df7e523a4340 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.172791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 126/241] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/users => generated 293 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:41.174843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ff56-a8ef-4423-9247-50f6e43255d3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.175047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ff56-a8ef-4423-9247-50f6e43255d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.175432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ff56-a8ef-4423-9247-50f6e43255d3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.182806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 127/242] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:41.188960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e4684-d295-4043-99ef-4566435324bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.189427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e4684-d295-4043-99ef-4566435324bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.189865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e4684-d295-4043-99ef-4566435324bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.209367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.222271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f55e4684-d295-4043-99ef-4566435324bb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:41.225798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f55e4684-d295-4043-99ef-4566435324bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:41.250915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08c2e7-6827-482d-83e9-eb46db7d117f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.316177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.317192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.317368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.317816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.319241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:41.319777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.322265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.339296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 116/243] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:41.351049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.438953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 128/244] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:41.449355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684bdae-d08a-4f42-b265-985c3e9803d1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.449886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684bdae-d08a-4f42-b265-985c3e9803d1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.450343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684bdae-d08a-4f42-b265-985c3e9803d1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.453556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 129/245] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:41.458832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.459340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.459760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.483403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:41.485540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:41.488164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.611242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4ba7e-788b-40be-b288-475c70ab9058 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.612209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 130/246] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:41.613817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.663443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.664581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.664993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.665394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] PATH_INFO: `/v3/users/f8909191b1a84434a8f398d4099658da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.682893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] RBAC: Authorizing `identity:get_user(user_id=f8909191b1a84434a8f398d4099658da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.684652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.686884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 117/247] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/users/f8909191b1a84434a8f398d4099658da => generated 297 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:41.688280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.704096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.827706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a9598-3e61-4bf8-8eef-d32583c431e6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.877903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.878767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.879053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.879346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.884388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a47206-6297-4612-8bde-0cbee3839da2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.888639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.894974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:41.904027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:41.906108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:41.908114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 131/248] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:41.909875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.978101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.981732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.982519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.983387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.012817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:42.033618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:42.035364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:42.036905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:42.043286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 118/249] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:42.045346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.146069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd10c-1bd1-497a-9a4b-fb0d35c648e5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.239242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.242096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.243322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.243912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.245419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.345744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e51065-8706-4e1f-b9b5-ed019e669b54 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.378270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.385929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.388929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0506d9-a4f9-4f91-a062-a47e0a13c28a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.439457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.440546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.441033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.441519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.443000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:42.443555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.454526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.473975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 119/250] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.474923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.601048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 132/251] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:42.603507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90389-8359-4915-8dff-1c2aa9bc38c6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.604014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90389-8359-4915-8dff-1c2aa9bc38c6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.604509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90389-8359-4915-8dff-1c2aa9bc38c6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.608212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 133/252] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:42.610317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d90389-8359-4915-8dff-1c2aa9bc38c6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.652569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c6d7-b186-4cc6-8c03-2f0d2a3ce854 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.714900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.742350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.743757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.744335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.744820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.747155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:42.747742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.752139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.778815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 120/253] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.780795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.838670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.848734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d90389-8359-4915-8dff-1c2aa9bc38c6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.917925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.919539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.920044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.920611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] PATH_INFO: `/v3/users/8a372fb582674bb896fa44dce4b1b192` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.923229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] RBAC: Authorizing `identity:get_user(user_id=8a372fb582674bb896fa44dce4b1b192)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.926083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f78c83-6ade-4403-b43d-43950aef085a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.929311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 134/254] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/users/8a372fb582674bb896fa44dce4b1b192 => generated 297 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.934513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.934513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.934513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.963741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:42.966103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:43.007022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c02f0-7185-44be-ac38-6ba62d080baa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.079150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.081444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.082277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.083242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.085928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.089599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.093612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 121/255] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.096173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.104576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 135/256] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:43.105228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.219360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.294830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.304667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880b09a-4359-41a1-9e49-a5668934022e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.323076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16346e-cf83-41ae-88ec-b152d0e1c515 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.344118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.345337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.345337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.345646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.346917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.348439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.354454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 136/257] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.354454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.409512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.412294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.413299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.414124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.417046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.420631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.425315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 122/258] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.432334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.474308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841f9c5-f64d-433c-8f9d-8f5c1feecc7b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.546322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.548143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.548636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.549121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.550749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=f8909191b1a84434a8f398d4099658da) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:43.551290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.554097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.565718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 137/259] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/role_assignments?user.id=f8909191b1a84434a8f398d4099658da&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=c85d184365d949ca99a130b6fff5392f => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.567433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.635462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.705163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fdbfd-ef9c-47e2-b5c8-efc598d7a843 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.763412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.786116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.788066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.788780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfba68-4afa-420b-8957-85b77848c37c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.789231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.790026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.811400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:43.827019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:43.829398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:43.831248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:43.835816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 123/260] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:43.837823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.875111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.878683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.879092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.879585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/8a372fb582674bb896fa44dce4b1b192/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.889535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, user_id=8a372fb582674bb896fa44dce4b1b192, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.891655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.918140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 138/261] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:43 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/8a372fb582674bb896fa44dce4b1b192/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:43.922716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.047436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.073635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bee436-2a05-4294-b9cf-14e489c2e3e9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.143255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.143963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.144254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.144533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.145333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:44.145646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.147126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318c9b3-fa01-414b-ba09-c206ddf01fd6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.168612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 139/262] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:44.170771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11d953-d3e1-4edd-a7ab-a4272144d417 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.174275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570e2f7-3132-47ff-8d03-09a186217622 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.174680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570e2f7-3132-47ff-8d03-09a186217622 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.174922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570e2f7-3132-47ff-8d03-09a186217622 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.176782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 140/263] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:44.178843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570e2f7-3132-47ff-8d03-09a186217622 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.252305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.253612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.253929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.254346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.328388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.357439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.359173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d563a-76ee-498b-9b46-c6e15bd132b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.423827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570e2f7-3132-47ff-8d03-09a186217622 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.545005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.548643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.549714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.554501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 124/264] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:44.555942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] PATH_INFO: `/v3/users/0eec17e4b52b4bb8bdf212b568c50f07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.559810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.561447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] RBAC: Authorizing `identity:get_user(user_id=0eec17e4b52b4bb8bdf212b568c50f07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.565096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.566002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.566717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.572780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 141/265] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/users/0eec17e4b52b4bb8bdf212b568c50f07 => generated 283 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:44.575286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.608199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:44.611298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:44.613234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.862014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 125/266] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:44.866996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.924326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.938219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb2fc6-87af-469e-9f6d-c5663e6caafd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.047512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.049251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.050113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.051033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.053699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.056697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.060424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 142/267] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:45.062919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.149788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e487-95e2-4ad8-8861-71b1c5d736d2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.216364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.217792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.218526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.219196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.232250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.263611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ac57a-309b-4b93-840b-044d131c14a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.330040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.331968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.332670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.333345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.352660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.354039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:45.355449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2da81ebc-84e5-45b6-b266-dbff7ac54e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:45.370113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:45.372076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:45.373929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:45.378338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 143/268] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:45.381103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e8e6-59a4-400e-a480-2645ea9fccdb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.381864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e8e6-59a4-400e-a480-2645ea9fccdb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.382664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e8e6-59a4-400e-a480-2645ea9fccdb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.385293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 144/269] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:45.387042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e8e6-59a4-400e-a480-2645ea9fccdb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.393286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.566135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 126/270] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:45.566135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.585876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e8e6-59a4-400e-a480-2645ea9fccdb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.631473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.632462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.632937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.633425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.634544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=0eec17e4b52b4bb8bdf212b568c50f07) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:45.635061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.636886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342457df-e490-4e23-a418-90b30b61ad15 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.646898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 145/271] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/role_assignments?user.id=0eec17e4b52b4bb8bdf212b568c50f07&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:45.649445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.649951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.650449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.675518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:45.676955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:45.681572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.699959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799dbd1-991c-429f-aabc-7836efc15266 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.731777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.781144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.781847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.782132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.782427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.783220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:45.783532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.785012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.803882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 127/272] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:45.804773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.833340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 146/273] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:45.834768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.976802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa4c6-cd18-4ccb-b6e7-5e18bdf03025 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.985795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8f27-79ae-4c96-bc1a-659beda80ee4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.022424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.023362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.023975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.024267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] PATH_INFO: `/v3/users/80e228431d0c4af692777b8f4c1ba926` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.036152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.037059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.037733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.038617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.038921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.041232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.042526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] RBAC: Authorizing `identity:get_user(user_id=80e228431d0c4af692777b8f4c1ba926)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.043338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.044606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.045416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 128/274] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.046323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.053131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 147/275] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/users/80e228431d0c4af692777b8f4c1ba926 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.054141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.116356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.191083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280cf9-4e10-49fd-942a-891e2de730c5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.203955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c88f-4649-4a8e-a744-25fb88f54d12 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.244177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.244860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.245254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.245541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.246571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=0eec17e4b52b4bb8bdf212b568c50f07) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:46.246895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.251085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.251738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.251934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.252189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.252462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.254153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.256378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.258914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 129/276] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.259835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.264995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 148/277] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/role_assignments?user.id=0eec17e4b52b4bb8bdf212b568c50f07&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.268409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.350891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.412523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc9774-e713-473b-b983-6e8760bafc29 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.413390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dd93d-5b73-4e3f-aa7e-dbcafdd4c1f8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.426799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.458923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.459629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.459909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.460820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.461048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, user.id=80e228431d0c4af692777b8f4c1ba926, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:46.461360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.464214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8c3bd-3073-4577-a41f-2e24efd7cfa8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.471200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.471901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.472185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.472463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.473138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 130/278] 10.222.0.77 () {56 vars in 1308 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/role_assignments?user.id=80e228431d0c4af692777b8f4c1ba926&scope.system=all&role.id=8569379203d74bee9bdcc46f1e112633 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.473255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.474522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac8e9-e1b4-47a9-a9a7-6f777afca83a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.474712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.474826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac8e9-e1b4-47a9-a9a7-6f777afca83a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.475075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac8e9-e1b4-47a9-a9a7-6f777afca83a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.476837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 131/279] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:46.482670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079192-5011-430b-8d08-a6bfb7e572cc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.482992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079192-5011-430b-8d08-a6bfb7e572cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.483445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079192-5011-430b-8d08-a6bfb7e572cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.508307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-31079192-5011-430b-8d08-a6bfb7e572cc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:46.511643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-31079192-5011-430b-8d08-a6bfb7e572cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:46.517800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 149/280] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:46 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:46.603793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 132/281] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:46.628179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.714116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a94541-1ae1-4c6d-b98a-ea36df6b739c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.714526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a94541-1ae1-4c6d-b98a-ea36df6b739c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.714695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a94541-1ae1-4c6d-b98a-ea36df6b739c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.716332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 133/282] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:46.723872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.724552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.725047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.757382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:46.760463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:46.784986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0964e-4673-4a80-800f-d9dc76ceb909 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.815143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.843189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.844691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.845211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.845721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.868801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:46.879125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:46.880110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:46.880901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 150/283] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:46.885431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 134/284] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:46.890587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.923132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.992442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.022196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47e80-d612-495c-86c9-acd05ae31772 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.091969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.092673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.092966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.093249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.094842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:47.095157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.096588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.114769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 151/285] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:47.136121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.152023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac6a7-3e86-4958-89e3-bf4d654a848f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.237917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.238766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.238968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.239070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.239296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.258328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:47.266857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:47.267819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:47.268629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:47.284128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 135/286] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:47.292584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.387918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedb4f-9146-403b-8207-687e757ad6cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.460686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.461594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.461874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.462147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] PATH_INFO: `/v3/users/f8909191b1a84434a8f398d4099658da` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.464281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] RBAC: Authorizing `identity:get_user(user_id=f8909191b1a84434a8f398d4099658da)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.465754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.468003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 152/287] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/users/f8909191b1a84434a8f398d4099658da => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:47.476525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.540774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.556022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5cedc-e717-4511-aca2-ba15eef6174c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.634489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.635258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.635472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.635778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.636623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:47.636930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.638456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.661238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 136/288] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:47.673167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.711983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38de33-c347-46c9-90f6-7fd537898320 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.736153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.800814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.807680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.808180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.808693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.811074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.814274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.822960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 153/289] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:47.832374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.860136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689315f-8dba-438a-a08e-27ac8ce450a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.898923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.899883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.900164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.900440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] PATH_INFO: `/v3/users/8a372fb582674bb896fa44dce4b1b192` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.901896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] RBAC: Authorizing `identity:get_user(user_id=8a372fb582674bb896fa44dce4b1b192)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.903461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.905589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 137/290] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/users/8a372fb582674bb896fa44dce4b1b192 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:47.923255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.957851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.055219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1ede-5a38-45b0-b5e8-cf71b7f6f943 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.099010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.099684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.100116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.100392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.176493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb675-22ba-42cd-bdc2-018a34cdd30f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.182393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.185059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.222029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.267782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.268085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.268568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.268673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/f8909191b1a84434a8f398d4099658da/roles/c85d184365d949ca99a130b6fff5392f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.271090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, user_id=f8909191b1a84434a8f398d4099658da, role_id=c85d184365d949ca99a130b6fff5392f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.272525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.314828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 154/291] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:47 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/users/f8909191b1a84434a8f398d4099658da/roles/c85d184365d949ca99a130b6fff5392f => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:48.321174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 138/292] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:48.329577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.400031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.562774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cac7ac-12ff-450c-a027-0e3f274411e5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.636647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.637471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.637925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.638267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.641156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.642296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32be3f-2276-47d8-8191-fe8ff3923558 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.644596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 155/293] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.650815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.849640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693789d7-ffed-41c8-ab1c-2b076d1f7a09 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.885911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.904490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.905307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.905610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.905935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.907523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=8a372fb582674bb896fa44dce4b1b192) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:48.907839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.909358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7962f55-5f85-4751-996b-200fc9058d6a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.920349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 139/294] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/role_assignments?user.id=8a372fb582674bb896fa44dce4b1b192&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=8569379203d74bee9bdcc46f1e112633 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.465427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5e36-4f74-4074-bd0f-5c9d7b9a6fd8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.466153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5e36-4f74-4074-bd0f-5c9d7b9a6fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.466153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f5e36-4f74-4074-bd0f-5c9d7b9a6fd8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.467962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 156/295] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:49.474891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.475329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.475729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.507255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:49.507861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:49.610323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f034-088e-4603-9f2b-5ed3d58dde70 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.610799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f034-088e-4603-9f2b-5ed3d58dde70 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.611176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f034-088e-4603-9f2b-5ed3d58dde70 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.612723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 157/296] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:49.621458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb442e-3542-49a8-9832-4aa15c5cc60c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.621458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb442e-3542-49a8-9832-4aa15c5cc60c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.621458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb442e-3542-49a8-9832-4aa15c5cc60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.633862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 140/297] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:49.654773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.681169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dbbb442e-3542-49a8-9832-4aa15c5cc60c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:49.683847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dbbb442e-3542-49a8-9832-4aa15c5cc60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:49.817509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f96a4-d740-4d17-beca-ca6b74f5dd30 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.843640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.864192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 158/298] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:49.866692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897d76-29d2-4b7b-a1c3-3eb8e9534302 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.867164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897d76-29d2-4b7b-a1c3-3eb8e9534302 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.867655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff897d76-29d2-4b7b-a1c3-3eb8e9534302 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.870188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 159/299] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:49.889274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.889744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.890181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.890453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.892168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.892971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.893633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.914577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:49.917145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:49.919160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:49.928259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:49.930065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:49.931621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 141/300] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:49.934954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.978836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.048011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 160/301] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:50.049432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.133145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e8a7e-b631-47ce-ad2a-4d630f2217ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.189451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.190872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.191701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.192272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.194584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.197235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.207049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 142/302] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:50.207049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.207370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c534ad-7b21-4c66-9eff-953563238126 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.251041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.251841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.252187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.252529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.253380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:50.253757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.255222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.266470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 161/303] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:50.266945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.293806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.306141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.369388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e746-7d2c-4639-9c71-d7ccd5a182bf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.411806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b553d0-221e-4074-b91c-4d35a112ae78 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.416872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.418028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.418611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.419207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.434735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:50.443594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:50.445405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 143/304] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:50.446989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.463955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.464775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.465044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.465285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] PATH_INFO: `/v3/groups` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.466015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.467681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.496092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 162/305] 10.222.0.77 () {60 vars in 1134 bytes} [Mon Jul 18 16:23:50 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:50.496092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.597006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f347-3d46-45c0-a6c5-9c946258ae5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.606745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.656901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.658651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.659084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.659600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] PATH_INFO: `/v3/users/7848ecd70a2d452e81b502c6a99d899a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.666098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.680728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] RBAC: Authorizing `identity:get_user(user_id=7848ecd70a2d452e81b502c6a99d899a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.683126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.686506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 144/306] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/users/7848ecd70a2d452e81b502c6a99d899a => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:50.703595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.706579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8fd38-cb00-49de-ac2b-1b3114878218 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.781292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.782531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.783059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.783576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.785081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:50.785665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.788520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.805943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 163/307] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:50.817084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.915968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c55-cc5a-43b2-a61d-eca7f4b716cb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.996295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.997938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.998463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.998978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.000696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, user.id=7848ecd70a2d452e81b502c6a99d899a, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:51.001277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:51.004496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:51.015759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 145/308] 10.222.0.77 () {56 vars in 1308 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/role_assignments?user.id=7848ecd70a2d452e81b502c6a99d899a&scope.system=all&role.id=c85d184365d949ca99a130b6fff5392f => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:51.026727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea305-5c64-4143-a1ca-5cf8d5318030 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.103520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.105145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.105738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.106234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] PATH_INFO: `/v3/users/80e228431d0c4af692777b8f4c1ba926` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.108769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] RBAC: Authorizing `identity:get_user(user_id=80e228431d0c4af692777b8f4c1ba926)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:51.111900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7834b68-7d28-4d89-bb31-9addc27b630b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:51.115302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 164/309] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/users/80e228431d0c4af692777b8f4c1ba926 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:51.125319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:51.211742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:51.342300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7325b7-0ce2-4459-9cb4-2681044b697d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.398078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0568d-c873-479e-b7c3-0eced5057078 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.398423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0568d-c873-479e-b7c3-0eced5057078 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.398699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0568d-c873-479e-b7c3-0eced5057078 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.401614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 165/310] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:51.406333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.406674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.406934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.415240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.416162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.416640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.416640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] PATH_INFO: `/v3/system/users/80e228431d0c4af692777b8f4c1ba926/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.419934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=80e228431d0c4af692777b8f4c1ba926, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:51.424616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:51.429928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:51.431253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:51.444513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 146/311] 10.222.0.77 () {58 vars in 1334 bytes} [Mon Jul 18 16:23:51 2022] PUT /identity/v3/system/users/80e228431d0c4af692777b8f4c1ba926/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:51.453349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:51.534651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 166/312] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:51.555265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:51.720101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:51.731317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaad21f-697b-495b-8605-e1c46383fdfc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.774249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.775120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.775407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.775682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.788480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:51.797534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:51.799236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:23:51.800617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 147/313] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:51.806684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:51.922340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101939b-75a4-433a-8d4a-3eb216c8ab2a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.962094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.962826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.963108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.963386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.964161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:51.964469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:51.965893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:51.981219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 167/314] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:51.982773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.100660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bada1-4766-4e0a-8a13-3a16dbafe805 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.139485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.140953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.141272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.141551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] PATH_INFO: `/v3/users/f8909191b1a84434a8f398d4099658da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.142945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] RBAC: Authorizing `identity:get_user(user_id=f8909191b1a84434a8f398d4099658da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.144439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.150572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 148/315] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/users/f8909191b1a84434a8f398d4099658da => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:52.153258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.167362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.286589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e561-3e08-4115-8b96-754b8d71adda None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.354917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.356051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.356546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.356995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.496153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.499058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.520011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.716670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 168/316] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:52.723145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.752430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.847729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f7a1-b3dc-4343-bfe8-2efbd6cabf11 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.888016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.889267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.889754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.890896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.892578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.894328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc8ff0-03a6-489b-9f42-1cc4959ae3b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.901590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 149/317] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:52.904165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.111808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1884b-1f18-4308-975b-ae4adf19db06 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.135293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.186314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.187976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.188476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.188956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.190663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9, user.id=f8909191b1a84434a8f398d4099658da) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:53.191222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.193999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a70a00-fb84-4acf-8fb1-3e7137af0713 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.206461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 169/318] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/role_assignments?user.id=f8909191b1a84434a8f398d4099658da&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=c85d184365d949ca99a130b6fff5392f => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:53.628405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65bcd-ef61-4623-83b9-06f8dda9e510 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.629022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65bcd-ef61-4623-83b9-06f8dda9e510 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.629527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65bcd-ef61-4623-83b9-06f8dda9e510 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.632574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 150/319] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:53.641309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32a99-1fed-4e4c-9cc7-d72df6660af2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.641819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32a99-1fed-4e4c-9cc7-d72df6660af2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.645280 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f32a99-1fed-4e4c-9cc7-d72df6660af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.664715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e3f32a99-1fed-4e4c-9cc7-d72df6660af2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:53.665937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e3f32a99-1fed-4e4c-9cc7-d72df6660af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:53.756348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 170/320] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:53.779892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e65bcd-ef61-4623-83b9-06f8dda9e510 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.953825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.979338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e65bcd-ef61-4623-83b9-06f8dda9e510 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.012600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ba28c-20b3-48bd-9285-0b6b193ef04f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.012979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ba28c-20b3-48bd-9285-0b6b193ef04f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.013260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ba28c-20b3-48bd-9285-0b6b193ef04f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.015099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 171/321] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:54.023201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.023814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.024099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.042882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:54.044103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:54.047327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.050104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.050642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.051136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.053386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.058520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.062362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 151/322] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:54.067809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.164272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 172/323] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:54.187314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.197561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350cbf7-b22b-4463-818f-d4807ee488e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.238857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.239460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.239742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.240017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] PATH_INFO: `/v3/groups` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.240727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.242328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.243447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.278446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 152/324] 10.222.0.77 () {60 vars in 1134 bytes} [Mon Jul 18 16:23:54 2022] POST /identity/v3/groups => generated 226 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:23:54.351033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfd90a-52b6-47ed-a1ea-a7e263bca0bd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.397659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.398538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.398823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.399222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.412669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b7a2c8b6-0ed9-42aa-a409-8073be956809 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:54.420882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:54.423673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:54.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:54.430329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 173/325] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:54.431269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.456785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.512041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a15c-1738-4494-83a1-1944f35a8b7f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.512422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a15c-1738-4494-83a1-1944f35a8b7f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.513323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a15c-1738-4494-83a1-1944f35a8b7f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.516482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 174/326] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:54.524939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.525471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.525962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.548042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:54.549288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:54.605825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eda733-70fc-4e10-ab25-21f89a71d7ef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.655926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 175/327] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:54.669787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.670555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.670854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.671409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.672216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:54.672581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.674104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.685761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 153/328] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.691175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.692640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.726791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.837341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.871727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124de1-05db-488e-ab40-d4a615b8daaf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.907072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29c754-dc25-4fa1-9330-a149e8ca4e95 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.928957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.929993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.929993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.929993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] PATH_INFO: `/v3/users/7848ecd70a2d452e81b502c6a99d899a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.932252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] RBAC: Authorizing `identity:get_user(user_id=7848ecd70a2d452e81b502c6a99d899a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.933733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.935798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 154/329] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/users/7848ecd70a2d452e81b502c6a99d899a => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.942521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:55.001495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.004197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.005798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.006832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.037556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:55.045995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:55.048031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:55.049828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:55.051101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 176/330] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:55.058878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:55.125652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cc231-e1d1-4cc7-b00b-9331390c41eb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:55.205177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.207282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.208025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.208680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] PATH_INFO: `/v3/system/users/7848ecd70a2d452e81b502c6a99d899a/roles/c85d184365d949ca99a130b6fff5392f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.211730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7848ecd70a2d452e81b502c6a99d899a, role_id=c85d184365d949ca99a130b6fff5392f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:55.214726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:55.216488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:55.237316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 155/331] 10.222.0.77 () {58 vars in 1334 bytes} [Mon Jul 18 16:23:54 2022] PUT /identity/v3/system/users/7848ecd70a2d452e81b502c6a99d899a/roles/c85d184365d949ca99a130b6fff5392f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:23:55.259621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c439-6ef1-43c0-b15f-ff1439b39ff7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:55.334069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.334845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.335180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.335458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.336269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:55.336589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:55.338213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:55.350283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 177/332] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:55.363683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:55.393416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:55.585426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc72da-6423-4c8a-847e-d9474b4c79ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:55.653484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.655110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.655646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.656165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] PATH_INFO: `/v3/users/80e228431d0c4af692777b8f4c1ba926` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.658654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] RBAC: Authorizing `identity:get_user(user_id=80e228431d0c4af692777b8f4c1ba926)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:55.661298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d07707-1397-4217-a540-b2428ff7e113 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:55.665110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 156/333] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:55 2022] GET /identity/v3/users/80e228431d0c4af692777b8f4c1ba926 => generated 293 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:55.674491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:55.921869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051bbd-3f05-4f91-bbb0-52485136a98e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:55.992863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.994012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.994527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.994995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.996521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] RBAC: Adding query filter params (role.id=8569379203d74bee9bdcc46f1e112633, user.id=80e228431d0c4af692777b8f4c1ba926, scope.system=all) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:55.997042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:55.999966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a778d04-9b1b-42f8-9d9e-c72d873f40bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.023804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 178/334] 10.222.0.77 () {56 vars in 1308 bytes} [Mon Jul 18 16:23:55 2022] GET /identity/v3/role_assignments?user.id=80e228431d0c4af692777b8f4c1ba926&scope.system=all&role.id=8569379203d74bee9bdcc46f1e112633 => generated 510 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:58.770212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475bd38-1d8c-4341-ab12-ba2382dd43e7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.771047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475bd38-1d8c-4341-ab12-ba2382dd43e7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.771289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475bd38-1d8c-4341-ab12-ba2382dd43e7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.774378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 157/335] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:58.783971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.784732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.784900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.797451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.823667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:58.825728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:58.991095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 179/336] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:59.044252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd21ec-ccdf-4a1b-91e0-0e33ca0871da None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.044764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd21ec-ccdf-4a1b-91e0-0e33ca0871da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.045258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd21ec-ccdf-4a1b-91e0-0e33ca0871da None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.048859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 158/337] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:59.050682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.059766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bc684-48b8-4a4f-9289-12d4dadd2ee2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.060270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bc684-48b8-4a4f-9289-12d4dadd2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.060772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bc684-48b8-4a4f-9289-12d4dadd2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.104455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5f0bc684-48b8-4a4f-9289-12d4dadd2ee2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:59.106973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f0bc684-48b8-4a4f-9289-12d4dadd2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:59.347240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 159/338] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 16:23:59.353501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77039809-d404-4cbc-99aa-01692fbaa322 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.354004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77039809-d404-4cbc-99aa-01692fbaa322 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.354518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77039809-d404-4cbc-99aa-01692fbaa322 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.356658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c60c4-710d-4764-a493-7fcfafe86f8c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.357596 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 160/339] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:23:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:59.366117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.366397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.366737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.413811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:59.416462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:59.423921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.441894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.460507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.461874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.462369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.462890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.508213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:59.533151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:59.534131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:59.534977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:59.535576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 180/340] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:59.536987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.594768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 161/341] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:59.596284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.823591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a790-01e3-4424-adb6-c608973e6e99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.874232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1be89-86de-4574-baf4-6654ab232043 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.890109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.890990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.891263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.891540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.913017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:59.921354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:59.923607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:59.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:23:59.929993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 181/342] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:59.933791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.960671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.961327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.961603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.961911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.962711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:59.963016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.964472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.980254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.988402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 162/343] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:59.989366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.066181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.180856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64553a52-2fa6-4d95-96b3-d67c1acb813b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.198040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798387c4-af3e-448d-9e30-c9d833c06a55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.228438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.229256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.229544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.230460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.245648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:00.257477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:00.259420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:00.260375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:00.266566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 182/344] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:00.268306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.283728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.284826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.285264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.285734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.287056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:00.287628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.291329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc95cc-0f25-476a-b6de-d95b6b3d4a8a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.314101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 163/345] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.316718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9af59-5f31-46a1-ae54-a17e55f17b2c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.317214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9af59-5f31-46a1-ae54-a17e55f17b2c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.317711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9af59-5f31-46a1-ae54-a17e55f17b2c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.321722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 164/346] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:00.323523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9af59-5f31-46a1-ae54-a17e55f17b2c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.460078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.486030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaadab-d1f1-40d7-8030-3ee80f4d6051 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.518446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.553678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9af59-5f31-46a1-ae54-a17e55f17b2c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.558968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.560091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.560548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.561050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.662961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.664277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.664782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.665271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.666909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:00.667460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.673937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.703164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 165/347] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.705105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.720154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.722771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac9d2e-d317-4269-90bc-6e48dc543455 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.938385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2de-43cf-4d2f-acfc-d74d6cd3fdcf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.944316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 183/348] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:00.945753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.950898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.951332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.981041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:00.982927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.983211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:00.997686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.997953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.998764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.999400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.000456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.084584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 184/349] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:01.085796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.103792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.106505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b2be4-a310-4a9e-86c4-70805e317a8c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.240432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834cd8d-f6e5-4171-8160-ccc04414bb4d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.267073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 166/350] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:01.269110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a310-9943-4c3a-bce0-4a1641554f22 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.269509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a310-9943-4c3a-bce0-4a1641554f22 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.269866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a310-9943-4c3a-bce0-4a1641554f22 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.272285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 167/351] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:01.273580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a310-9943-4c3a-bce0-4a1641554f22 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.303653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.304350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.305920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.305920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.322907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.327460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.406780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.408301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.470125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a310-9943-4c3a-bce0-4a1641554f22 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.516010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.517505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.517505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.517505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.518709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.520351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d6430-e68a-4578-9af0-3516d793d63f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.527436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 168/352] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:01.530980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2670f91-8d7f-4502-808c-7c5c7466ade5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.530980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2670f91-8d7f-4502-808c-7c5c7466ade5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.530980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2670f91-8d7f-4502-808c-7c5c7466ade5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.533038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 169/353] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:01.534102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2670f91-8d7f-4502-808c-7c5c7466ade5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.582438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 185/354] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:01.582438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.650400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.679253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.695984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2670f91-8d7f-4502-808c-7c5c7466ade5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.751378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ea47-af9e-46c3-9378-5ea31ce5c442 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.759397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.760215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.760481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.760759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.777583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c8916130-4b2f-4654-901c-0a79cac652b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:01.787982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:01.788992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:01.789798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 170/355] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:01.791830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b899-9096-4247-8297-0d875c42bca3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.792147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b899-9096-4247-8297-0d875c42bca3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.792433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b899-9096-4247-8297-0d875c42bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.807267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.808362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.808766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.809149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.810678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.814086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.817956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 186/356] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:01.818180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1aa4b899-9096-4247-8297-0d875c42bca3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:01.818345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.819637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1aa4b899-9096-4247-8297-0d875c42bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:01.920314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 171/357] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:01.921903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.922191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.922561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.948457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:01.949727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:01.981611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fdf88-7931-46bf-8572-e3e2110d5a8c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.999948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.036518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.037345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.037712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.037906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.047094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.048629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.050973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 187/358] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.051504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.063882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 172/359] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:02.064756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.089290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.196696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dfba-4ad0-4505-aa3f-ee943738b81b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.244203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87827508-2b32-4f77-9acd-ddddad6e5220 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.244809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.245771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.246047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.246343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.247762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.249873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.252155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 188/360] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.252414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.300663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.301468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.301884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.302292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.303273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:02.303676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.305300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.308775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.316513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 173/361] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.317951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.382701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295e1f-0cf0-4f7b-9eab-3272d3f641be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.404236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.420211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.421604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.421902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.422173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.423470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.424930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.426670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 189/362] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.427467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.443675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071053b-afc4-4f35-9014-66e06b827ff4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.485777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.486667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.486942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.487205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.499058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:02.507554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:02.509293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 174/363] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:02.510932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.551236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52b8a0-cced-4d23-866b-0ead616b740a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.562964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.604049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.605107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.605570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.606028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.607581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:02.607969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.610147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.619907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 190/364] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=8569379203d74bee9bdcc46f1e112633 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.620925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.680779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2715ae-c41c-4a0e-8493-bfba2a6690d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.744130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.745358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.745944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.746341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.750938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.768531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:02.782351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:02.783392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:02.784334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 175/365] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:02.787132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.805824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd34a-850e-4d4e-b159-b1078becc9b3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.869233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.870748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.871249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.871747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.873997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.876824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.880094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 191/366] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.881512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.925679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bdd76-06e2-4897-a258-c5f572b52945 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.935379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.978529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.979422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.979759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.980036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] PATH_INFO: `/v3/users/7848ecd70a2d452e81b502c6a99d899a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.981359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] RBAC: Authorizing `identity:get_user(user_id=7848ecd70a2d452e81b502c6a99d899a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.982866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.985289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 176/367] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/users/7848ecd70a2d452e81b502c6a99d899a => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.986109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.027632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadaeee-df09-49dd-8b0d-2d6127ea6d67 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.070043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.091250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.091922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.092200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.092474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.094064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.094381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.095904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.110980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 192/368] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.114119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.156835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73ed1-02c1-4ffe-9dc2-28282f153277 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.226886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.228025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.228583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.229054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.230517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.231041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.234079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.250799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 177/369] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.252379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.344497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c96352-972d-452d-986d-5c9beb64281d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.360540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.433853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.435704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.436407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.437652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.439821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.441220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.444140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.467651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 193/370] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.470144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.481651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed124-531c-4809-8f9a-f4b7c7d6f9a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.542812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.572447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.574294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.575015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.575825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.578752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.579551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.582848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.603197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 178/371] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=8569379203d74bee9bdcc46f1e112633 => generated 260 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.604990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.699286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9867eb-d15a-4878-8910-488adedeac6f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.782318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.783947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.784769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.785558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.787519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] RBAC: Adding query filter params (role.id=c85d184365d949ca99a130b6fff5392f, user.id=7848ecd70a2d452e81b502c6a99d899a, scope.system=all) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.788423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.791884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.817699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 194/372] 10.222.0.77 () {56 vars in 1308 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/role_assignments?user.id=7848ecd70a2d452e81b502c6a99d899a&scope.system=all&role.id=c85d184365d949ca99a130b6fff5392f => generated 510 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.819630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.879612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.010032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e5517-8f61-4172-8bb1-39e609bce09f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.040979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced31e2-1b4a-4684-ad6d-5f8a4026c91b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.111877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.113689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.114449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.115109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.157312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.158292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.158805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.159282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.262470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.265910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.320551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.350547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.400093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.402782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.475511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 195/373] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:04.484480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.670593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 179/374] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1061 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:04.675286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.695053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd845d-bcdd-4e87-8f31-34c69904d184 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.754557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.775099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.778093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.779018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.779828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] PATH_INFO: `/v3/projects/c66249a1b75d4939b69e5011863c8709` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.783340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] RBAC: Authorizing `identity:get_project(project_id=c66249a1b75d4939b69e5011863c8709)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.786556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.791220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 196/375] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/projects/c66249a1b75d4939b69e5011863c8709 => generated 344 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:04.798095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.934345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.956871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2c83-8964-4f55-9543-89f344f060b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.060783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.062311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.062829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.063309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.068991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.070970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf3709-9dbd-4f05-b329-5eda51e51180 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.071799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.075673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 180/376] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:05.114901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.154316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.156800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.157579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.158277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] PATH_INFO: `/v3/groups/34025d70811542dbb759b0af66919c38` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.168361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] RBAC: Authorizing `identity:get_group(group_id=34025d70811542dbb759b0af66919c38)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.171765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.175890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 197/377] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/groups/34025d70811542dbb759b0af66919c38 => generated 229 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:05.196131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.384911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13999b7-c0dc-433e-8e83-4602ae081896 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.396601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.502179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.506243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.507369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.510279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.513120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.517158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.525881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 181/378] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:05.537975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.542031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd83929-0b3c-47aa-833f-c25e1376b28d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.657408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.688502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.690437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.695176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.696086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.697796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] RBAC: Adding query filter params (group.id=34025d70811542dbb759b0af66919c38, role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=c66249a1b75d4939b69e5011863c8709) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:05.699992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.703044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51824-4652-4d90-b937-3188f362e344 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.726782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 198/379] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/role_assignments?group.id=34025d70811542dbb759b0af66919c38&scope.project.id=c66249a1b75d4939b69e5011863c8709&role.id=54d519e3caa64c32a9930f925d38c156 => generated 260 bytes in 532 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:05.795865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea9558-db23-49ed-873b-812c3b04c318 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.796849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea9558-db23-49ed-873b-812c3b04c318 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.797415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea9558-db23-49ed-873b-812c3b04c318 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.806946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 199/380] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:05.814682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.815281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.815746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.847180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:05.849747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:05.858955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809106d5-f5be-4a3e-a423-34062ea2ba0c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.938016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.939711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.940770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.946449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.946449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:05.946449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.946449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.957229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 182/381] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 260 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:06.011194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 200/382] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:06.035337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.192680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:06.216560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454ae0-230a-47b5-a827-3f6de84e537c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.270262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.271545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.272067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.272548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.285978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:06.294776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:06.295859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:06.297413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:06.300673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 183/383] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:06.307255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.388360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:06.462040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783c77f-5e82-4c8a-b781-61dde55dbe04 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.525574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.526461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.526833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.527228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.528250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:06.528621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:06.530038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:06.541274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 201/384] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:06.547142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.739089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f22d2-419a-46bf-a7fa-a12b6f198c54 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.817788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.818713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.819028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.819300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.823338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:06.824865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:06.827089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 184/385] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:06.849415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.087669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463992b3-4e0d-4cfb-abc1-e9fec273411d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.127996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.184090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.185668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.186166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.187098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.189739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.192538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8df1c-4b69-40ca-898a-d4c7f5fab6eb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.202888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 202/386] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:07.207618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.252206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.319033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf90a-c9bc-4f05-910d-17074503f8cb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.320552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf90a-c9bc-4f05-910d-17074503f8cb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.321336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf90a-c9bc-4f05-910d-17074503f8cb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.324800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 203/387] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:07.335329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.336277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.337069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.367158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:07.369838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:07.415758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4ddf-232e-47bc-9cf8-c1c4edb69cbf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.484944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.487092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.487926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.488708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.492803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, group_id=e8b777cc8f1f420db17f8c579f40f5f8, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.495914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf89620-ed10-44c0-b734-ba0efc79fefa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.524201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 185/388] 10.222.0.77 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:07 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:07.529793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402225e-36b9-4e9d-86f8-86fdc852c979 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.531907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402225e-36b9-4e9d-86f8-86fdc852c979 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.532147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402225e-36b9-4e9d-86f8-86fdc852c979 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.536242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 186/389] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:07.545753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.546298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.546879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.562514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 204/390] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:07.585147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:07.587434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:07.629711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.728693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 187/391] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:07.793664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.819942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.928579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e11c-4b75-4a3b-b7e6-7bd102746278 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.960904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:08.023740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.025878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.026802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.027598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.052333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202c2de-ee52-4b31-a8a9-d884f4905467 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.074538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:08.095008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:08.096772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:08.098348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 205/392] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:08.110369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.163526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.165426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.165921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.166384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.195529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:08.213099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:08.215742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:08.217357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:08.222551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 188/393] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:08.231291 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.293613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d754-aa0c-48ec-adf1-11b3d224e66b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.345392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.346567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.346976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.347419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.348711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:08.349212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.360424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.387698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 206/394] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:08.400071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.415131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:08.477102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a9-66bf-4c27-a2f2-9d050c01df4f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.548957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.550107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.550628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.551108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.552562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:08.553007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:08.553168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.557547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.576895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 189/395] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:08.586239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.758647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64679220-798a-4c05-b42c-e46de65f3e0c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.834299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.835877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.836363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.836865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.840512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.842007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.844356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 207/396] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:08.853904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.938038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9f71-10d2-44a8-90e5-bf957290ea74 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.999184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:09.012317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:09.014689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.015206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.015701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.018083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:09.021387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:09.025437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 190/397] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:09.033425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:09.225200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304b20-db0f-4f82-b355-ffdbb1603231 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:09.265788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4737694-beeb-4dad-becf-4b0fd014af10 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:09.278899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:09.292647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:09.294471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.295522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.296732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.300366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:09.303798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5391935-20eb-4d90-8470-a74c39eb27fd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:09.315327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 208/398] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:09.315327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9fbf1-5661-4ce7-91f6-f9114153a27d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.315327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9fbf1-5661-4ce7-91f6-f9114153a27d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.317540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9fbf1-5661-4ce7-91f6-f9114153a27d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.324065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 209/399] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:09.325928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9fbf1-5661-4ce7-91f6-f9114153a27d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:09.349862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:09.351488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.351956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.352419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.355303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:09.357985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbed2a-8332-479e-9302-7f363a18f383 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:09.366643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 191/400] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:09 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:09.369075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.369554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.370075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.402444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:09.404730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:09.530833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:09.536512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9fbf1-5661-4ce7-91f6-f9114153a27d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:09.549892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 192/401] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:09.551794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:09.633287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:09.634894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.635378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.635904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.640044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, group_id=e8b777cc8f1f420db17f8c579f40f5f8, role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:09.651425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:09.686514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 210/402] 10.222.0.77 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:09 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/345ac5f37e244b4faa681fbd95404a3d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:09.686514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:09.762222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:09.803587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb67797-caaf-416d-ba99-8aff5fba2824 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:09.887458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:09.889057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.890190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.890813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.894811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] RBAC: Authorizing `identity:create_grant(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4, group_id=e8b777cc8f1f420db17f8c579f40f5f8, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:09.897086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a90ad-b750-4672-9a68-9c1ed204da7d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:09.934560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 193/403] 10.222.0.77 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:09 2022] PUT /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:09.937196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd67a-d2b6-43ae-b255-317f9969a2c1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.937750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd67a-d2b6-43ae-b255-317f9969a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.938196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd67a-d2b6-43ae-b255-317f9969a2c1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.939514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498b98-7bcc-4630-bb28-28134eb2ffa4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:09.940351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 194/404] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:09.953079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.953723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.954171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.978813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:09.981162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:10.008086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.009594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.010069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.010570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.026793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:10.037328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:10.038290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:10.039132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:10.045834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 211/405] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:10.048142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.081310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.139125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 195/406] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:10.204900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.281124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914263-5360-4546-91ae-554b0f0227e3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.286811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.358704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.359721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.360155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.360601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.361982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:10.362796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:10.365417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:10.387865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 212/407] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:10.397162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.520796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311a453-25b5-492c-b2d7-91b525a7ca66 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.599485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.600338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.600624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.600906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.618893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3c9c433c-b3bd-439f-861d-2fc64625b07a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:10.628814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:10.631149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 196/408] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:10.632619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966afea-8251-483e-af84-9a3f645e6e94 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.641709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966afea-8251-483e-af84-9a3f645e6e94 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.641709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966afea-8251-483e-af84-9a3f645e6e94 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.641709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 197/409] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:10.641709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966afea-8251-483e-af84-9a3f645e6e94 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.660443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97206969-a92f-4d20-923a-d44d6e41e144 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.719663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.721438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.722137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.722930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] PATH_INFO: `/v3/groups/34025d70811542dbb759b0af66919c38` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.724415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.725449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] RBAC: Authorizing `identity:get_group(group_id=34025d70811542dbb759b0af66919c38)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:10.728574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5fd07-f04b-4026-a26a-7c8414e9051a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:10.732520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 213/410] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/groups/34025d70811542dbb759b0af66919c38 => generated 229 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:10.735540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.736482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.737159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.768893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:10.771418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:10.795869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966afea-8251-483e-af84-9a3f645e6e94 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.797083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.840916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.841860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.842324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.842911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.843987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:10.844451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:10.846137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:10.856964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 198/411] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:10.858887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.914607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 214/412] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:10.916374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.984655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f6e69-48bd-4977-a0f0-772c66adce86 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.993525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.027816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.028808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.029199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.029536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] PATH_INFO: `/v3/projects/c66249a1b75d4939b69e5011863c8709` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.031562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] RBAC: Authorizing `identity:get_project(project_id=c66249a1b75d4939b69e5011863c8709)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.033303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.035577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 199/413] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/projects/c66249a1b75d4939b69e5011863c8709 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.040620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.109651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb0ec7-ec0a-4d47-8f23-cc0a58354119 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.153611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.154260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.154652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.155021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.156537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.158222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.160663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 215/414] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.161761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.169543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08c27b-5cca-49c6-b10e-a76138077c45 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.184438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.215946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.217177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.217646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.218250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.231801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:11.240790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:11.241760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:11.242658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:11.245070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 200/415] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:11.246159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.287820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.289121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e3b1d-0302-4133-99e5-28157f8cac07 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.339692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.340613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.340968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.341299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] PATH_INFO: `/v3/projects/c66249a1b75d4939b69e5011863c8709/groups/34025d70811542dbb759b0af66919c38/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.345335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c66249a1b75d4939b69e5011863c8709, group_id=34025d70811542dbb759b0af66919c38, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.347001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.374448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 216/416] 10.222.0.77 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:11 2022] PUT /identity/v3/projects/c66249a1b75d4939b69e5011863c8709/groups/34025d70811542dbb759b0af66919c38/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:11.374448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.465998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfb5a4-01ca-4148-b68c-bd5f960f57de None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.478970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.518358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c242ad-11f0-42d4-92e5-5599a266e6e7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.567690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.569158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.569554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.569903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.570803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:11.571182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.572929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.593450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 217/417] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.594760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.599876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.603582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.604313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.610449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.615451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.618716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.622901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 201/418] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.633078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.744754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a5beb-0134-4e62-a52a-da9f737e8568 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.817295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.818943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.819495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.820067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.852876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.940957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753c9e8-81e7-44f4-a165-8c610e79ce9d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.943715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.993747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.994937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.995235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.995626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.998548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.998809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=982e44e99cf14fc0aa3c23e386bfc6a9, group_id=e8b777cc8f1f420db17f8c579f40f5f8, role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.001923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.003076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.026670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 202/419] 10.222.0.77 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:11 2022] PUT /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9/groups/e8b777cc8f1f420db17f8c579f40f5f8/roles/345ac5f37e244b4faa681fbd95404a3d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:12.254465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 218/420] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:12.259894 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.419234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ac7af-2c43-431d-b449-960581866a2e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.462756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.463642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.463981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.464258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.465557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.467074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6870-50c9-4b06-bdb9-c54dd62ef53f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.469774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 203/421] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:12.475962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.595851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486cb51-031e-4594-9dca-1f318a76072f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.596213 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486cb51-031e-4594-9dca-1f318a76072f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.596778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.597732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486cb51-031e-4594-9dca-1f318a76072f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.602363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 204/422] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:12.613793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc5a4-9572-4949-9a11-9bf78fbf4bab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.614084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc5a4-9572-4949-9a11-9bf78fbf4bab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.614367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc5a4-9572-4949-9a11-9bf78fbf4bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.656562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-acdfc5a4-9572-4949-9a11-9bf78fbf4bab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:12.658221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-acdfc5a4-9572-4949-9a11-9bf78fbf4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:12.677701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5d190-43b2-444e-9d7d-21b1ab26fd0c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.687675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.727394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.728286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.728584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.729142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.730551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.732721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3d52a-9776-425c-973f-fc03c889610d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.739792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 219/423] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.739792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeaf5d-5337-420b-a2c1-06909ad1387d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.739792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeaf5d-5337-420b-a2c1-06909ad1387d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.739792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeaf5d-5337-420b-a2c1-06909ad1387d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.739792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 220/424] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:12.743608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeaf5d-5337-420b-a2c1-06909ad1387d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.754683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 205/425] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:12.756599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.757150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.757625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.790935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:12.792581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:12.878628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 206/426] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:12.879964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.898513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeaf5d-5337-420b-a2c1-06909ad1387d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.911666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.949118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.949845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.950123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.950393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.951520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.951840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.954793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.963277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 221/427] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=8569379203d74bee9bdcc46f1e112633 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.964162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.027891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.058510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269248a4-7611-4e8e-bc2a-45f0baf2e710 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.144965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.148118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.149334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.150893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.167859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525f152-79a4-488e-9f2e-aef3fd194c6c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.184824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:13.195509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:13.196486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:13.197645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:13.201823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 207/428] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:13.208992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.241523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.242348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.242673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.242956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.256997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:13.267460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:13.268432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:13.269267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 222/429] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:13.275403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.320445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.345343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138fe1d-54e6-4405-be97-c5f8cb3f37cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.384772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.385909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.386445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.386973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.387976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:13.388517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.390188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.400026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 208/430] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:13.407305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.418069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941f98-bacd-4e48-a610-312f56a9115a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.428945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.464705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.465680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.466156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.466780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.468351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:13.468976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.470672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.481363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 223/431] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:13.487885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.533508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10e565-4918-4990-9540-9516a0f48830 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.573330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.574090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.574395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.574708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.612869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.622522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e36c6-1335-4fb0-9f61-b451335da382 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.655923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.657448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.671774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.672871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.673373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.673862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.674612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.759485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.761515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.784687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 209/432] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:13.785926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.898269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.914539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 224/433] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:13.918654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.941771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3d409-8d3d-474e-a3b8-90c6b6fcaa72 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.972681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.013567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.015679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.016638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.017390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.020148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.023715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.027892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 210/434] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:14.034886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.085207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f115fc-c86f-4cc5-ae46-afd3ab966a79 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.127470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.128466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.128750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.129024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] PATH_INFO: `/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.130439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] RBAC: Authorizing `identity:get_project(project_id=d62d49bdea5e4b7d995cb8fecdd9d8c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.132066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.138649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 225/435] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/projects/d62d49bdea5e4b7d995cb8fecdd9d8c4 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.139901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.191827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504a385-dcbb-4be6-be34-c44c0e4b1602 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.216838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.232367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.233531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.233531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.233802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.235103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.236582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57670a6-b45f-4ce4-8ac8-b2b318dd0016 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.243365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 211/436] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.244878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f28c-4799-401f-884b-5dbcf6e37cf0 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.245398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f28c-4799-401f-884b-5dbcf6e37cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.245398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f28c-4799-401f-884b-5dbcf6e37cf0 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.250569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 212/437] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:14.250569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f28c-4799-401f-884b-5dbcf6e37cf0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.265849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3166c7-a0d4-4be2-ad98-187e3436bc70 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.303975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.304874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.305162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.305442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.306786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.307859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.308282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc70aa-4772-4285-b0f8-72f41505266f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.315547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 226/438] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.315547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.315547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.315547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.331049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:14.332311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:14.372419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f28c-4799-401f-884b-5dbcf6e37cf0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.407673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 227/439] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:14.408910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.420543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.421275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.421555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.421825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.422709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:14.423016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.424494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.431925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 213/440] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.456426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.493410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.553039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bbb36-d3cc-420a-9484-83cd198ad7a0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.603992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a229-f4eb-40f8-ae9f-80e1d658f812 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.609650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.610457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.610816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.611187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.612134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=8569379203d74bee9bdcc46f1e112633, scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:14.612519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.614146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.622631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.624628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 228/441] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=d62d49bdea5e4b7d995cb8fecdd9d8c4&role.id=8569379203d74bee9bdcc46f1e112633 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.654638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.656243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.656854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.656942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.670162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:14.678377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:14.679794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:14.680716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 214/442] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:14.686051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.816955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c578a1-6b51-43ba-8740-affd095bc655 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.844579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.858794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.859496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.859783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.860070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.860827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:14.861143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.862600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.872458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 229/443] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.878969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.002565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da46d77-a7b4-48f7-b13c-57318c87827e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.045025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.045727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.046528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.046812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.103196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.149695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.152392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca62885-d717-4a6c-ab5a-09487f8cf343 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.321344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 215/444] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:15.328173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.492597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3092-cc2f-4ad3-b658-64a8ac20e64d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.492895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3092-cc2f-4ad3-b658-64a8ac20e64d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.493171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3092-cc2f-4ad3-b658-64a8ac20e64d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.494672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 216/445] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:15.501246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.501538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.501812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.511884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee51007-127f-4547-a6fe-aa4aa1ea84ba None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.520218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:15.521468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:15.562748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.563637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.563918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.564192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] PATH_INFO: `/v3/projects/c66249a1b75d4939b69e5011863c8709` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.565482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=c66249a1b75d4939b69e5011863c8709)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.566996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.574950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 230/446] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/projects/c66249a1b75d4939b69e5011863c8709 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:15.576839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.601578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 217/447] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:15.625526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.627889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.650848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.705117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c1e2-d905-42e5-b7ac-b42ceeb2b7d2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.745653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.746487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.746775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.747053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] PATH_INFO: `/v3/groups/34025d70811542dbb759b0af66919c38` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.748340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] RBAC: Authorizing `identity:get_group(group_id=34025d70811542dbb759b0af66919c38)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.750033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.751654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 231/448] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/groups/34025d70811542dbb759b0af66919c38 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:15.754164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8bc49-b644-42e3-a8be-5f6d2e7f54c0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.758139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.791574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.792394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.792684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.792962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.805412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:15.814397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:15.815433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:15.816261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 218/449] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:15.821879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.894493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.898582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d97c9-bd81-4250-a275-231cf97c7a68 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.924225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.937939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.938598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43d7e2-257d-490a-9823-3d7092de515c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.938886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.939258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.939623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.940684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] RBAC: Adding query filter params (group.id=34025d70811542dbb759b0af66919c38, role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=c66249a1b75d4939b69e5011863c8709) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:15.941129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.943157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.949922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 232/450] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/role_assignments?group.id=34025d70811542dbb759b0af66919c38&scope.project.id=c66249a1b75d4939b69e5011863c8709&role.id=54d519e3caa64c32a9930f925d38c156 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:15.979283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.979981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.980264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.980539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.981330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:15.981642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.983134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.993000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 219/451] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:15.999489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.121547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19234f7f-0fb3-49a5-87ae-a7cded486c94 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.162804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.163478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.163761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.164038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.226565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.241106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.242623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.355363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 233/452] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:16.361412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.467335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfa4b-6663-41ff-87c0-79d7cdef9373 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.492615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.505156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.506037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.506318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.506628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] PATH_INFO: `/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.507914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] RBAC: Authorizing `identity:get_project(project_id=982e44e99cf14fc0aa3c23e386bfc6a9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.509394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.511285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 220/453] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:16 2022] GET /identity/v3/projects/982e44e99cf14fc0aa3c23e386bfc6a9 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:16.517397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.608735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.624704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d4ffe-c790-4ec5-8cb9-41c7250a0900 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.661581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.662486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.662774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.663053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] PATH_INFO: `/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.664316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] RBAC: Authorizing `identity:get_group(group_id=e8b777cc8f1f420db17f8c579f40f5f8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.665778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3a90-01c5-447a-a8a4-57740f5f1ea0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.667517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 234/454] 10.222.0.77 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:16 2022] GET /identity/v3/groups/e8b777cc8f1f420db17f8c579f40f5f8 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:16.673928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.781019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622693ca-2c89-4dd6-a2b3-8d30b4228080 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.817057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.817788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.818109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.818465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.819442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] RBAC: Adding query filter params (group.id=e8b777cc8f1f420db17f8c579f40f5f8, role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:16.819851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.821423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312821e6-1216-47a7-a59a-bda9ef3833e8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.828202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 221/455] 10.222.0.77 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:16 2022] GET /identity/v3/role_assignments?group.id=e8b777cc8f1f420db17f8c579f40f5f8&scope.project.id=982e44e99cf14fc0aa3c23e386bfc6a9&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:20.099781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee0f7-7142-420e-b56e-84cb192effc0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.100865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee0f7-7142-420e-b56e-84cb192effc0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.101311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee0f7-7142-420e-b56e-84cb192effc0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.103322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 235/456] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:20.108852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b4c5-5b58-4634-a38c-1514e811e732 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.109725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b4c5-5b58-4634-a38c-1514e811e732 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.110142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b4c5-5b58-4634-a38c-1514e811e732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.134160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-19e8b4c5-5b58-4634-a38c-1514e811e732 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:20.135731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-19e8b4c5-5b58-4634-a38c-1514e811e732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:20.206610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.236103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 222/457] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:20.277867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee0f7-7142-420e-b56e-84cb192effc0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.438719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee0f7-7142-420e-b56e-84cb192effc0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.481721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14952-5a69-457b-9a8e-54338d05bcb3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.482260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14952-5a69-457b-9a8e-54338d05bcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.482797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14952-5a69-457b-9a8e-54338d05bcb3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.489478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.490329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.490572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 223/458] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:20.490675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.490898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.495340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7599-7e57-4af4-b5dc-ab54f75251d9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.495834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7599-7e57-4af4-b5dc-ab54f75251d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.496323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7599-7e57-4af4-b5dc-ab54f75251d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.511684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.515180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:20.541410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:20.542367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:20.543642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:20.543642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:20.543642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:20.543642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:20.546888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 236/459] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:20.554861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.555221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-484c7599-7e57-4af4-b5dc-ab54f75251d9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:20.557419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-484c7599-7e57-4af4-b5dc-ab54f75251d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:20.571023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.694552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 224/460] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:20.694552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927964a8-f1e4-48bd-86a1-bab35687ef27 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.694552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927964a8-f1e4-48bd-86a1-bab35687ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.694552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927964a8-f1e4-48bd-86a1-bab35687ef27 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.694552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 225/461] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:20.704397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.704828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.705454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.729840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:20.731159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:20.786141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e33f2-35de-47ff-b8be-a02273a3f1f5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.832425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 226/462] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:20.833858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.866231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.868169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.869108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.869897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.878965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:20.880040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.883394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.889963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.918653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 237/463] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:20.920044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.971245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9ed7f-7b2a-44da-bdaf-eeb4e805ddc1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.002789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.024229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.025256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.025664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.026103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.041520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.043727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.047599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6665b-068d-4c66-8ffc-3451bcb5c013 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-06721cd4-e8a9-4971-a10c-ca3526b719ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:21.068692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:21.070444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:21.072181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:21.072181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:21.072181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.072181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:21.072181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 227/464] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:21.078690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d56aef-0e6a-4ba1-b62e-fe4ea5874f5a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.078690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d56aef-0e6a-4ba1-b62e-fe4ea5874f5a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.078690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d56aef-0e6a-4ba1-b62e-fe4ea5874f5a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.083001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 228/465] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:21.084130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d56aef-0e6a-4ba1-b62e-fe4ea5874f5a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.097173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.098002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.098286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.098600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.112445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.113964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:21.135211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:21.142125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 238/466] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:21.143308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.252966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.262301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d56aef-0e6a-4ba1-b62e-fe4ea5874f5a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.315387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327d80-33eb-4d06-8987-41e47878121e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.320082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.320787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.321096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.321501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.322787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.325351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87096ef4-f328-415c-9ee9-5dc5e703b541 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.347929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.366443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 229/467] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:21.366443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.366443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.366443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.385659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.386677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.387112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.387595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:21.387786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.388727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:21.389186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.389689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:21.390877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.412973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 239/468] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:21.414272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.531039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 230/469] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:21.532455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.574749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a0df-0d66-4e4f-8b60-fc29abdf3e5b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.636117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.636811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.637102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.637388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.638141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:21.638509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.640643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.666205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 240/470] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:21.667499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.712749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.740008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f777a-223f-4876-9bd0-fdc685c046d0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.792828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.793603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.794053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.794649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.796169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.798545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.819016 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90ffb-2ada-48f4-bfee-03d7456c98ba None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.834442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 231/471] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:21.834442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.903431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.906191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.906689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.907150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.928446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.931355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3de0058f-5217-4d57-b76e-14bbe1196c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:21.966759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:21.967735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:21.968592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:21.968592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:21.968592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:21.968592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:21.975113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 241/472] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:21.977515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b079c8-351a-487e-b5d7-d7fd4897b30b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.977974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b079c8-351a-487e-b5d7-d7fd4897b30b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.978466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b079c8-351a-487e-b5d7-d7fd4897b30b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.986977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 242/473] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:21.991881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.992340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.992797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.021385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:22.023608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:22.054014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6639d6-ef45-49ed-8b84-47cf6c343a0e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.113817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.115115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.115765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.116781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.117574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.119212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.133237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.147729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 232/474] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/users => generated 249 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:22.149147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.184813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 243/475] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:22.222215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.261010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.362742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db182-e69c-4960-89b4-6d83b09aed91 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.441214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.442248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.442997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.443747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.445297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:22.446098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.448978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.460788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214e60c-87c8-474a-a7e5-fb15be213782 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.480434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 233/476] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:22.488813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.533682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.535345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.536323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.537228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.539032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.542300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.554800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.sql.core [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:22.554800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:24:22.554800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [parameters: {'id': '515cdc204b424e6398e4cedc8664284c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:24:22.554800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80165) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:24:22.571329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result.read() Jul 18 16:24:22.605842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:24:22.607493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application yield resource Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise self.value Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application session.commit() Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:24:22.609728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:24:22.613457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:24:22.616893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result.read() Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:24:22.620374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [parameters: {'id': '515cdc204b424e6398e4cedc8664284c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:22.624477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:24:22.628083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:22.631337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 244/477] 10.222.0.77 () {60 vars in 1131 bytes} [Mon Jul 18 16:24:22 2022] POST /identity/v3/roles => generated 144 bytes in 408 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:22.631680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd308e6-e843-4db4-92e0-764c3bff60c3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.640694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.687615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.689719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.690195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.690721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.692019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.694903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.731325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 234/478] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:22 2022] POST /identity/v3/users => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:24:22.857906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.894828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279765-9d59-442f-a4c0-38ffddfc2573 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.965900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.968410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.969196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.969975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.991320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5bb179cc-7e08-4c4f-bd36-24a317df319f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:23.006867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:23.008569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:23.010118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:23.015019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 245/479] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:23.023709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.178158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a8b5-c1cb-4d6f-9d87-cf80b4f67b54 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.194995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:23.222318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.223035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.223320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.223593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.224361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:23.224681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.226180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22acf6-bd2b-48e8-8f9e-5a9e0295bdb3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.237393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 235/480] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:23.885958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddafe7-ef09-416d-9520-b254be2417ba None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.886528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddafe7-ef09-416d-9520-b254be2417ba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.886976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddafe7-ef09-416d-9520-b254be2417ba None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.890121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 246/481] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:23.896512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.897482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.897969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.919193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:23.921371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:24.017740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 236/482] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:24.061737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddafe7-ef09-416d-9520-b254be2417ba None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:24.220956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddafe7-ef09-416d-9520-b254be2417ba None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:24.277417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:24.278741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:24.279261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:24.279746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:24.299761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:24.308356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:24.309310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:24.310118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:24.317886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 247/483] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:24.319689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:24.471978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723f019-89c9-4250-b480-5b7b6a7a7bfb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:24.478008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:24.528310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:24.529018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:24.529302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:24.529578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:24.530954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:24.531271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:24.532782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:24.547149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 237/484] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:24.553032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:24.558988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:24.737673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3c044-2de1-46cc-94bc-016a9cd172b0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:24.812322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:24.813740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:24.814230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:24.814767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:24.850262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:24.853135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:24.901431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:24.902840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:24.903750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:24.903750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:24.903750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:24.903750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:24.904435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 248/485] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:24.909996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:25.075372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa93f2-c1bc-427b-9c1c-8706c0a4e4b9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:25.128670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:25.129762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:25.130251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:25.130757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:25.131710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:25.132222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:25.133835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527f6b9-0f54-422b-b886-a1f3c4a8983e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:25.156563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 238/486] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:25.162788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:25.303751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:25.418728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2662ac-0e5c-4423-bc0e-951cda6c9056 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:25.450115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1456e9-5581-40dc-8d3a-d24ab52d4196 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:25.450779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1456e9-5581-40dc-8d3a-d24ab52d4196 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:25.451249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1456e9-5581-40dc-8d3a-d24ab52d4196 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:25.452987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 239/487] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:25.459419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:25.460050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:25.460524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:25.497984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:25.500240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:25.503351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:25.504631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:25.505119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:25.505614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:25.521727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:25.525410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:25.562888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:25.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:25.575187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:25.577820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:25.581951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 249/488] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:25.587979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:25.748619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 240/489] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:25.787552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:25.833084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c62fd4-cb40-4f8a-aaf5-c747d79965a5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:25.893626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:25.930928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:25.931699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:25.932269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:25.932745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:25.933825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:25.934363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:25.936394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.001392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fd539-c132-45c6-889d-8c96ae4891a9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.013671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 250/490] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:26.020046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.081601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.083199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.083851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.084338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.105897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c6bbe2a5-ec23-42e0-acea-d0ba2bba5e13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:26.122806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:26.125098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:26.126756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 241/491] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:26.131146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d63d6-c5fd-4196-b616-5fee3636f287 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.131146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d63d6-c5fd-4196-b616-5fee3636f287 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.131146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d63d6-c5fd-4196-b616-5fee3636f287 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.132936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 242/492] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:26.134548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d63d6-c5fd-4196-b616-5fee3636f287 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.172790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.178869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac0d16-2dd8-4562-a9dc-ecbe89084001 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.222647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.223612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.224134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.224555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.278060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.301975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d63d6-c5fd-4196-b616-5fee3636f287 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.315560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:26.317104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d349caf-fa27-48f0-8a02-49f54d392650 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.349386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.350483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.350899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.351522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.352551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:26.353077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:26.354950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4bcef-1589-4c35-bc96-53f9b94f6606 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.367659 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 243/493] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:26.375064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.375512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.375969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.402520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:26.404545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:26.479488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 251/494] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:24:26.482121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81624b2-2888-4563-99ff-136dbac62584 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.482815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81624b2-2888-4563-99ff-136dbac62584 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.483846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81624b2-2888-4563-99ff-136dbac62584 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.490471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 252/495] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:26.491769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81624b2-2888-4563-99ff-136dbac62584 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.528052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 244/496] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:26.528052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.554706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.617491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.658022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81624b2-2888-4563-99ff-136dbac62584 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.680420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119efeb-03a7-4460-9ee1-1e4531b191b1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.696073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.696895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.697177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.697449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.709866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:26.711404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.720279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.721104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.721457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.721665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-946eafaa-74ad-485f-84cf-ce43cfdc78f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:26.732803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:26.734077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:26.735403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:26.735403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:26.735403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.735403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:26.735805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:26.735952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 253/497] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:26.736073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.736717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640f55-dbcf-45ab-91ad-81e0fca25f62 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.737203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640f55-dbcf-45ab-91ad-81e0fca25f62 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.737606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640f55-dbcf-45ab-91ad-81e0fca25f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:26.755846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:26.757499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 245/498] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:26.759143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.761200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d7640f55-dbcf-45ab-91ad-81e0fca25f62 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:26.762484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d7640f55-dbcf-45ab-91ad-81e0fca25f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:26.844209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.853851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 254/499] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:26.855413 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc4d-d79e-4711-aff5-4f49a68f8efc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.855580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc4d-d79e-4711-aff5-4f49a68f8efc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.855823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc4d-d79e-4711-aff5-4f49a68f8efc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.857465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 255/500] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:26.858000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dc4d-d79e-4711-aff5-4f49a68f8efc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.905898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85068f35-09ea-4e64-8375-124adaedecea None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.933108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.948421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.949231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.949506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.949779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.962379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:26.971564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:26.973168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:26.974065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 246/501] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:26.974926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.987504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dc4d-d79e-4711-aff5-4f49a68f8efc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.026038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.026757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.027046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.027332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.028079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.028394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.029913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545b03d-1462-4f45-9e17-66769c0e1f3a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.042578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 256/502] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.043984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.044283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.044560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.062839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:27.064099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:27.097434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aa5d7-a4f4-4b56-8e45-67cb1cf7e44d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.120000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.136523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.137192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.137468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.137738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.138508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 257/503] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:27.145525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.145525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.145825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.145977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.153840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 247/504] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.155570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.209025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.271712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75183e38-f776-46a2-8284-c3bb80147e14 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.280675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906c93-78bc-4c5a-99c4-555a14485f55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.309236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.310084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.310370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.310690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.318658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.319343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.319622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.319891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.320662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.320976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.322403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.324131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:27.332274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 248/505] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:27.332657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.334276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:27.336409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:27.337568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.337706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 258/506] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.337706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.392246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.462668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88495fc-ed11-47ba-a7e3-af511f900f0a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.468588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800e7a-4147-41c0-9535-95d1017fa705 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.477084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.501887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.502729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.503005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.503271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.509891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.511017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.511480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.511851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.512130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.513630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.520796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.522319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:27.532623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.533785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 249/507] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.535947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:27.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.541214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 259/508] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.546159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.667001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842af7-2beb-4402-85f4-673bfa8d59b0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.669043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.670129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7a193-9a09-4326-9987-ab7cd6024b88 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.708232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.709164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.709341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.709607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.709881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.710031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.710191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.710467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.711252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.711560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.713014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.729754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 260/509] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.729754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.731322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.732837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:27.752896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.754081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.755264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:27.755264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:27.755264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.755264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:27.758874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 250/510] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.759248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.759562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.855904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70799388-5c10-4886-a325-deb99e3e00f5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.872880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05838-08e0-4ef5-8ab8-ed281e447c82 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.895570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.896390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.896672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.896947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.910466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.911853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.914146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.914871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.915156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.915430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.916300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.916610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.918094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:27.931345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.932480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 261/511] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.935544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.961551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 251/512] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.962506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.981121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.034777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.061929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f448-ca5b-40f6-9ad9-4dd21a93c7b9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.082646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e55c8-a24e-4b10-8d87-cb5f989783ef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.103119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.103951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.104316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.104675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.105650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.106049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.107711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.121551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.122357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.122856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.123144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.129239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 262/513] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.130179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.139298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.140841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:28.159914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:28.161520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 252/514] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:28.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.241480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.257357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524f069-4888-4d3a-8c1f-4afd685626a7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.287126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4cd7b-5367-434d-b93f-028b37213fa8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.297625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.298302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.298616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.298895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.299629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.299937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.301351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.311558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.316094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 263/515] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.317185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.340147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.340823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.341149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.341427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.342682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.343047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.344434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.358787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 253/516] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.359539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.438976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94710a8-c6c0-4c73-b93a-3075edfc9fe4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.473866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b77fd2-e17d-45af-9fe4-aa6099c3cc6c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.478479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.479155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.479439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.479745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.516904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.517731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.518075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.518429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.519432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=5e2880530a3d43b482fb27ac23701293) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.519862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.522013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.528977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 254/517] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/role_assignments?user.id=5e2880530a3d43b482fb27ac23701293&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.530371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.571456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.574753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.577489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.702882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3f11-a9c2-4eea-baa0-d4b730e17b21 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.746237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.747450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.747941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.748432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.749794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.750337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.753048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.754482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 264/518] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:28.755770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.777267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 255/519] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.778858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.873821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.918176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.922180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86183dee-751e-4adf-bb11-ce4b8e072fab None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.970216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.970905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dee82-905b-4eb6-ac66-0d39d288dcf0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.970991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.971241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.971402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.972109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.973741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.014752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 265/520] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:28 2022] POST /identity/v3/users => generated 247 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:29.014752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.037616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.038769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.039188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.039521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.050898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.052822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:29.074671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:29.076400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:29.078720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 256/521] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:29.079829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.233635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b4945-f99a-43f7-8c45-4a712045d758 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.304185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a8b67-60f9-41ab-8c56-5d232437664c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.310678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487c3e-299b-4355-b748-505868d7248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.312089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487c3e-299b-4355-b748-505868d7248c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.312352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487c3e-299b-4355-b748-505868d7248c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.312628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487c3e-299b-4355-b748-505868d7248c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.327887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.328865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72487c3e-299b-4355-b748-505868d7248c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.332368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72487c3e-299b-4355-b748-505868d7248c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-72487c3e-299b-4355-b748-505868d7248c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:29.368943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:29.369863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:29.370897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:29.370897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:29.370897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:29.370897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:29.373315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 266/522] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:29.373315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487c3e-299b-4355-b748-505868d7248c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.386995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.388374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.389482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.389704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.392337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.408681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.412338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:29.444494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:29.446612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:29.448438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:29.453232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 257/523] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:29.460193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.518091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487c3e-299b-4355-b748-505868d7248c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.557666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.558528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.558874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.559216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.560251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:29.560593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.563257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.590446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 267/524] 10.222.0.77 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:29.590446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.673606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e191b-b1d5-43b7-8168-d160470cb926 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.700434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.742340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2649-ce8f-4fb4-9ca6-179c8bbe59c4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.749677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.751414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.752350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.753118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.755047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:29.755918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.758602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.784375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.784962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.785236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 258/525] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:29.785970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.786225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.787175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:29.787628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.789286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.791598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.811625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 268/526] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:29.812583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:29.857306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.945227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe1fec-9173-4e46-acb9-e11cd8ab9247 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.974706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.988356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.989769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.989769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.990060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.998167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.000317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.016017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07fb09-5593-4c88-bd8f-f6840ffc967e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:30.016278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:30.017622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 269/527] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:30.019350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.089675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.091375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.092019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.092725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.142070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12370177-d2d1-43a6-a3bb-dc6a0ecea021 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.181079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.182250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.182250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.182250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.238840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.242271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.243784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.263817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.265718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.304525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.390463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 270/528] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:30.391237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.462124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 259/529] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:30.463572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.510995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.524837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5433d34-d87a-4df5-bc4c-275542ecb32e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.575307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.576022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.576302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.577147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.578271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:30.578606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.581444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.596779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 271/530] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:30.602544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.642012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce21f5-57f7-4cac-814c-86a0f0dd8f5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.700135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.717751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.719178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.719739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.720208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.741696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.744672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.745138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebd56e-f254-4df5-9af9-94a17b5576c1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:30.783290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:30.784250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:30.785112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:30.785112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:30.785112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:30.785112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:30.788649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 260/531] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:30.790224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.793827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.795446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.795932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.796580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.813649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.816192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.820401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:30.843038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:30.843959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:30.844834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:30.844834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:30.844834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:30.844834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:30.847120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 272/532] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:30.848815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.985150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1114f-9b72-477a-acab-36c80bfa1ce2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.000425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126e0a3-44be-454c-b43a-f6f75c4048f0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.037335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.038819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.039277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.039745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.040697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.041172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.042768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.060421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 273/533] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.061666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.073121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.074668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.075283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.075913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.078372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=527930459ed340c0b09ee6c6da770a21) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.079138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.082527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.095126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 261/534] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/role_assignments?user.id=527930459ed340c0b09ee6c6da770a21&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.096801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.152006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.250655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.325071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b2576-e8c0-46a5-89fb-b7423c360783 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.365974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed0d2a-7a98-4c23-8080-910e93328a06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.400210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.401367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.401861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.402340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.403738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.404264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.408334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.410130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.411229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.411742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.412239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.420531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.422384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.431742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 274/535] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.438779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:31.441876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.443124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:31.444050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:31.446733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 262/536] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.456006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.594803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5dd4e-56c5-4c7a-b27e-3c1a481d9e8a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.597983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.637527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.638599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.639693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.640179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.641194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.641744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.643479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa58d18-4ae4-430e-a255-257810354b3e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.664573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 263/537] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:31.666438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0119-763f-4912-9402-fadedc0955b6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.666763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419df6-8d62-434b-a69f-332d202fc42b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.667191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0119-763f-4912-9402-fadedc0955b6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.667831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff0119-763f-4912-9402-fadedc0955b6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.669851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 264/538] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:31.675828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff0119-763f-4912-9402-fadedc0955b6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.700496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.731687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.732511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.733379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.733660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.743562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.745101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b4c5cdbf-384c-410f-85c8-2eb88cd75c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:31.765028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.766673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:31.768308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:31.770570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 275/539] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.773358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.774895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.775257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.798394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:31.799826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:31.865469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff0119-763f-4912-9402-fadedc0955b6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.914865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.915878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.916169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.916445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.917319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.917947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.920085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab991-ef5c-41ed-b6cb-95f3284b2868 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.927123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 276/540] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:31.928637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.931891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 265/541] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.937270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aee54-8192-404b-9bd8-1503ddc197da None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.937270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aee54-8192-404b-9bd8-1503ddc197da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.937270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aee54-8192-404b-9bd8-1503ddc197da None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.937270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 266/542] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:31.944320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.944675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.944879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.972970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:31.974810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:32.024074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.086579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 267/543] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:32.088301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.152391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562269b6-3258-43fd-adb4-0aaaa9b985c2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.162848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.200977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.202146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.202690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.203182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.206016 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.206617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.210249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.247763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 277/544] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.249406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.302008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d250d-950c-4a4d-932c-3f72a3f2229a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.375131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.376712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.377287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.377854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.400251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:32.417533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:32.419321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:32.421206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 268/545] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:32.424051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.468153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.473924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4ed0-df87-42a8-a561-289e423e71b4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.549661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.551060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.551675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.552568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.573282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374db7-731f-4d3a-8c52-5fcd47956dc1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.573673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:32.600553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:32.603358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.606087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:32.615559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 278/546] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:32.615559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.619177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.619954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.620403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.620771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.621725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=1747a2eb4850417d8d02a2b0a8d0cff7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.622113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.623777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.631020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 269/547] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/role_assignments?user.id=1747a2eb4850417d8d02a2b0a8d0cff7&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.631932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.696563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.912341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27062c0e-cfd2-4abe-90f7-ba388fa53d2a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.916474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.983513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.983996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6be5e-2cd4-438f-9226-0571784321d3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.985279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.986659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.987444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.988867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.989350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.991966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.007850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 279/548] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.016998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.095198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.096985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.097726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.098445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.100472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.101370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.112565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.140144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 270/549] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.149707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.192048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.213073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ee22-7547-4a67-80cc-a65e57fb58cd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.284993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.286556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.287124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.287640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.311452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.314466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:33.353147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:33.354106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:33.355032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:33.355032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:33.355032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.355032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:33.362640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 280/550] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:33.366201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.471138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b612-1e7b-4084-8d2f-881de3d859a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.555373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.556553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.557066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.557548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.573466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.575373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.579136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.586493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25e9a-cf90-4473-87b7-c6e9d6d9b4d5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:33.612771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:33.614393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:33.616069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:33.616069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:33.616069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.616069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:33.628357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 271/551] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:33.633932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.666685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.667953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.668632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.669243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.670892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.671592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.674059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.706453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.706926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 281/552] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:33.710368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.759061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11e88e-2f33-4f6f-b64a-97280aaec04b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.824019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.824775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.825161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.825548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.827027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.827415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.828981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.861340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 272/553] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:33.867046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.946671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d2a2-9af0-4bb3-8dfa-6eafc70f7a9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.019005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.057588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.059070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.059589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.060110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.090493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.096320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0b725893-936d-471b-908c-763fd2742dcc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:34.146899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.147885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:34.148804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:34.156318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 282/554] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.161980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddb86-fb57-4e8f-867c-91ce08d1abbc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.165008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cd59a-573c-413f-978f-76d7f8bcb912 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.165514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cd59a-573c-413f-978f-76d7f8bcb912 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.166027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cd59a-573c-413f-978f-76d7f8bcb912 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.171518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 283/555] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:34.173455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cd59a-573c-413f-978f-76d7f8bcb912 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.207606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.208436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.208720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.208995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.218209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.219747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0152c15a-e61c-48aa-b47d-56425f9a404a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:34.237712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.239965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:34.241912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 273/556] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:34.249168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.249168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.249168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.264926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:34.266221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:34.304524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.358833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 274/557] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:34.359782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.398146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cd59a-573c-413f-978f-76d7f8bcb912 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.445999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.447386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.447698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.447977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.448868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.449185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.451301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.457919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.481761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 284/558] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:34.483251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.583017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e739-0812-48d5-a6ec-d0fc53384244 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.605159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16cae6-df42-4eef-bc5f-80d7fd16b938 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.625466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.648315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.649426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.649916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.650375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.663095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.664362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.664923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:34.665036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.665517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.667309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.667953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.670915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:34.674768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.675799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:34.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:34.685420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 285/559] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.686875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.722750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 275/560] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.726818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.811011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd4262-c62b-4d70-9915-758e5d7fcc27 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.853225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.854248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.854752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.855319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.911262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.924251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd06bd-4d8e-4115-b8fe-5987b66ee54a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.926969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.945423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.947260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eae2d-22e6-4d4d-906b-b915bd108912 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.974337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.975526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.976012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.976503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.977910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.978496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.981106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.002321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 276/561] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.002321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.096249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 286/562] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:35.098037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792cca4-0d21-4df2-8992-8be14d1c5db0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.098395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792cca4-0d21-4df2-8992-8be14d1c5db0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.098798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792cca4-0d21-4df2-8992-8be14d1c5db0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.100385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 287/563] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:35.105254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792cca4-0d21-4df2-8992-8be14d1c5db0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.155817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95a3c-9514-4938-8599-77adf995a8a3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.201008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.202038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.202508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.202916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.209243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.217653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.219230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5fe45608-f269-45ae-83c3-20c62b45e357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:35.238752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.240539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 277/564] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.242154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.242877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.252537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792cca4-0d21-4df2-8992-8be14d1c5db0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.271340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:35.273431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:35.291442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.292259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.292541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.292813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.297878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.304843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.306368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:35.326545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.327698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.328785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:35.328785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:35.328785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.328785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:35.330783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 288/565] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:35.331683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.361340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 278/566] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:35.362295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.459757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275473b6-a246-42fa-aeb1-13ff2fea1bea None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.481018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d21b9-d995-4a01-a54f-040155ae85a3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.494612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.502997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.503816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.504112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.504938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.517321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.519306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.520252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.520943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.521227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.521505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.522253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.522603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.524060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbb77-62ab-4643-abbf-41f22d5cac72 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.537166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 279/567] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.547410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb082a-670f-44d8-b13b-f05f7428ac74 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.547410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb082a-670f-44d8-b13b-f05f7428ac74 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.547410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb082a-670f-44d8-b13b-f05f7428ac74 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:35.547970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.549817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.551977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:35.551977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:35.551977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.551977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:35.552486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 280/568] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:35.552486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb082a-670f-44d8-b13b-f05f7428ac74 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.552759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 289/569] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:35.552759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.563838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.666251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb082a-670f-44d8-b13b-f05f7428ac74 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.676788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d366c5-8432-46ef-8d40-e2555f773d6d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.713216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.713806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.714097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.714375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.715160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.715506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.717408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.721192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.722095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.722469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.722807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.731180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 281/570] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.732049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.739401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-877bc3d6-7de9-41e0-8b7a-fd7cdf6f4407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:35.748904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.749944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:35.750855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 290/571] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:35.752752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.753236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.753718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.773662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:35.774924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:35.814726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.818709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.855468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 291/572] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:35.856354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.858133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b041-096e-4511-8b55-da6948d644f3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.898750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.899662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.899980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.900256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.900985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.901309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.902739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.915967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 282/573] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.916602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.999733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cf690-2049-4f0b-b333-4a99608a7161 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.025024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526db1a0-f080-4865-8c5b-0c411dd28aab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.036664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.037476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.037762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.038041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.046159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.047687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.054170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:36.063096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.064067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:36.064962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:36.065392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.065392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.065392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.065392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.066188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 292/574] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.067551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.074661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.076141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:36.091950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.092995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:36.093963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 283/575] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.094826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.124942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.206495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fac0f-c8a7-44ed-be2e-7ca379d7976d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.216071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5cd3b-5524-4c35-b0f8-f69022d9fb4a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.244208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.244897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.245169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.245443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.246201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.246536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.247927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.253298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.254028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.254313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.254733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.258546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 293/576] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.259637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.266816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:36.274969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.276629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 284/577] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.277504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.332591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.376383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.384266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba672a-be3b-4643-9022-becf2536b749 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.392250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778bdf1-cdfa-4f7e-9ad6-6b0c7f550143 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.423329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.424139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.424421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.424695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.432580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.433616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.434063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.434521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.436386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.436561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.436888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.438872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.439222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.453854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 285/578] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.454781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:36.456462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.458057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 294/579] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.461182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.573729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f8e-ed5a-4b33-ba6c-e486331e0c4f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.577372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd9969-8161-4274-8cc0-329b62c1e595 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.592119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.612874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.613675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.613954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.614225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.617272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.618089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.618575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.618978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.619955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.620381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.622058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.626157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.627687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.649185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 286/580] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.649185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:36.649731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.651737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 295/581] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.655626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.659794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.767116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b817d-5b13-4dc3-b30b-70de5c6080d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.767714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0984c1-44d5-411c-a026-5664f8dce394 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.808871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.809320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.809780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.810056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.810256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.810476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/5e2880530a3d43b482fb27ac23701293/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.811257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.811667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.812638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.813032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.814711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.823709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 287/582] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.824643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.833243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=5e2880530a3d43b482fb27ac23701293, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.834929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.850072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 296/583] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:36 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/5e2880530a3d43b482fb27ac23701293/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:36.850994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.877066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.923745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.983671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06beaefc-c3e6-484d-a528-2682864b8ae8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.004178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614578-73c9-44ac-8493-c63bcd263cd0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.039402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.040761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.041246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.041710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.050312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.051055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.051335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.051607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.052425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.052729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.054360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.061667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.064589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.073927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 288/584] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.073927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:37.098976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:37.100068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:37.101144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:37.101144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:37.101144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.101144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:37.101377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 297/585] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:37.102771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.184349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.209428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c98d5-5818-42ba-878f-76112932cc29 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.257301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1ebf-7190-4870-a68a-32bbde5068eb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.271530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.281360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.282459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.282861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.283258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.284485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.284998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.287381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.307535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 289/586] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.308971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.321806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.322870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.323298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.323693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.338450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.341098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:37.379409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:37.387438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 298/587] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:37.389768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.497332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b666aa-239b-4809-95c1-abb85b0da9ae None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.531483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4beee-84e3-4984-a831-ea82ae50a442 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.561253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.562364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.562877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.563383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.565018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=71aecafdbfad40d4979f6f6358a58109) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.565589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.569002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.579863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.590642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.590864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 290/588] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/role_assignments?user.id=71aecafdbfad40d4979f6f6358a58109&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.591038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.591038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.591038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.591347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.591467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.591467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.610930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 299/589] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.611985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.626824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.652723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.789020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e504e9d-a601-4e3c-8327-706ca9a2df61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.822275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b236-a91f-4c46-97ed-97732c512688 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.859862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.860721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.861100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.861507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.862302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.862941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.864610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.882552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 300/590] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.888252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.914119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.916976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.918193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.918944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.940456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.944617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:37.974697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:37.976296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 291/591] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:37.978675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.038875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.071777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22652985-9514-4e5c-b585-e1967e992e86 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.111477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.123117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.125736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.127152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.128221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.142213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.143849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:38.159791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.161766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 301/592] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.163767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.172417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ec955-984c-4bea-9b22-b4aa6c93de36 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.252064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.252872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.253152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.253425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.266906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.268545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:38.285066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.286902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 292/593] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.289483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.335864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe405-6d1d-4e9d-9746-57240f53ef35 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.376402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.377101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.377386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.377665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.378584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.378896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.380394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.396308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.410654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e193fe-4ec8-41d2-94eb-5bbba5edd95b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.414235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 302/594] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.414512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.447152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.452125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.453126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.453987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.455325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.456410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.456911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.458616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.473722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 293/595] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.475130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.532615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55773f0-acc8-4d71-ad5f-fc33829ec231 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.575093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.575794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.576076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.576901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.577752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.578062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.579576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.600752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 303/596] 10.222.0.77 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.602083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.608015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f221d0-96e9-49da-a6c8-73cf7324fbc8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.667087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.677042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.679916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.680677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.681363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.704303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.707829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.727678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20a9ba-9dbe-4a0d-90fa-dd640c430521 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:38.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.749219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.751099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:38.751099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:38.751099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.751099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:38.755092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 294/597] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.757097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.778608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.780160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.781156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.781156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/527930459ed340c0b09ee6c6da770a21/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.799510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=527930459ed340c0b09ee6c6da770a21, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.801175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.820112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 304/598] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:38 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/527930459ed340c0b09ee6c6da770a21/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:38.821832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.859737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.956025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acf61d-aad9-4d09-81c2-14e2c586e3cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.966076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.972088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc7edf-5df6-4ddb-82b9-cc29349e0894 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.005318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.006441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.007114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.007601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.008658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.009207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.010890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.025361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 305/599] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.035241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.054495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.054495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.054495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.054495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.074509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.077782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:39.113645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.114673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.115554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:39.115554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:39.115554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.115554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:39.116333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 295/600] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.126244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.252110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa1ac-0db0-4c5a-8919-8db98c8cdee4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.294620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.296534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.297051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.297531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.306191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.308020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:39.325070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.326035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:39.327066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:39.332981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.333349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 306/601] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:39.337875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.344434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564861-f3f5-49a1-9951-108a7fdb1460 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.372619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.412754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.413920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.414442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.414927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.416366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.416898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.419582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.443895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 296/602] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.452708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.469265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f747368-0fd6-40a6-ac8c-73c199e43bb6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.516242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.517302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.517788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.518261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.519311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.519830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.521633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ecdd9-51c6-49f5-8099-d3855ebd8920 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.540580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 307/603] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.542679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c85fa7-ab2a-4ad7-be6d-29eda2f07ca4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.543179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c85fa7-ab2a-4ad7-be6d-29eda2f07ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.543727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c85fa7-ab2a-4ad7-be6d-29eda2f07ca4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.547416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 308/604] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:39.548727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c85fa7-ab2a-4ad7-be6d-29eda2f07ca4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.675813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.689566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bef92-f61f-42e2-9a46-a111c714a025 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.696834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c85fa7-ab2a-4ad7-be6d-29eda2f07ca4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.738549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.739839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.740253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.740667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.758756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.759984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=bd1cd1cd83fa43cab0dcbd1698c715fd, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.760227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.760719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.761209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.762115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a9f63-9898-40b2-bf66-758ff047fa69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.775257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.778024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.790744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 309/605] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:39 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:39.792810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.793238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.793613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:39.808955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.811268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:39.813006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:39.819190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 297/606] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.824285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.835817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:39.838626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:39.902843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.058971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 310/607] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:40.097187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.121650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c664-d76f-48ff-aa15-2a4c4191a2bd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.199761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.222518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.223636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.224118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.224586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.225602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.226477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.228224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.255563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 298/608] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:40.267463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.284517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cf84e-5196-4ac0-b4c3-554db40ab132 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.352148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.353366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.353857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.354328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.368321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:40.377034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.378110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:40.378991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:40.386563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 311/609] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:40.389362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.447454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.480088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93de57-4fe9-4809-be35-f06165081a81 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.517855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980cabf-32b9-4179-a1a7-7f0446f50721 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.532311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.533355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.533640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.533913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/1747a2eb4850417d8d02a2b0a8d0cff7/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.553256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.562740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=1747a2eb4850417d8d02a2b0a8d0cff7, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.564514 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.564684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.565410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.565813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.566196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.567265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.567691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.569392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40f80-dfbc-45dd-b1ac-7d285f2e8d6e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.585921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 312/610] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:40.585921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864699b-b94e-4e2d-ab34-29c1ddfdbb8f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.585921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864699b-b94e-4e2d-ab34-29c1ddfdbb8f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.585921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864699b-b94e-4e2d-ab34-29c1ddfdbb8f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.587495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 299/612] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:40 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/1747a2eb4850417d8d02a2b0a8d0cff7/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:40.589881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 313/612] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:40.590543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.591536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.592540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.592896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.615559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:40.616998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:40.706931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 314/613] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:40.731909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.745708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109b3d-259f-4a97-aa84-d4802586b0c3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.786044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.800026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.800834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.801119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.801574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.815374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.817637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:40.840403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.841391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.842353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:40.842353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:40.842353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.842353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:40.842353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 300/614] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:40.849246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.927875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dddb4b-f104-4aac-b18c-03f1e8d23fb8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.968224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.984905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d610-18dc-4ebd-8ada-4b9a3ec3e108 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.020167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.021466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.021968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.022553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.030160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.030874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.031154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.031428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.032544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.032677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.034093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.039466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:41.048266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.049310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:41.050157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:41.053190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 315/615] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.057755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 301/616] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:41.059722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.064495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.118392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.194963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f9-e816-4c97-87dc-0412471c53fd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.200321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9640791-67ef-4f11-902b-addf1deda276 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.235651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.236449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.236738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.237013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.238055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.238207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.239778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.243871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.245055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.245542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.246147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.251314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 302/617] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:41.257153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.257383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.259194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:41.276524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.277566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:41.278559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:41.286291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 316/618] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.288337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.337342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.376580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.379777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5b56-dc0e-4b00-8bf6-10c69acba423 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.414611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d06e3c-18a0-468a-8999-56f7e7ffa074 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.431392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.432224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.432510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.432788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.457542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.459002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.494950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:41.495557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.497526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.500589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:41.500589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:41.500589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.500589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:41.501313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.501313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.501313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.501313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.502020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 303/619] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.502199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.504381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.505550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.536503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 317/620] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:41.548237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.770145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae688f30-1806-416f-ba03-3506dbd8a14a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.774990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.783113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abc78d-8e8d-4d72-b70e-292418a8da79 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.892998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.894524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.895602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.897519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.900418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.901485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.905838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.913014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.914340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.915032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.915659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.943618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 304/621] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:41.952442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.985056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.115588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.119623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.224772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b9de-6036-4acf-bcb8-06e21d37e76a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.282965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.283788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.284070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.284342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.302611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.304143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:42.332982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:42.335187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:42.337443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 305/622] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:42.342445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.395086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.420485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 318/623] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 873 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:42.431529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.432550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.473362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccda58d-1532-4dfb-8ad3-8021e8373424 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.516487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.517271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.517558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.517856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.520368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.522189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.523754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.562990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 306/624] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.579526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.659361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15c578-1093-4e34-85cc-9641da4ecc1f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.704239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.705344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.705643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.705922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.712855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de81fd0-9c52-44f0-8718-543424e44575 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.720676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.723274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1236e178-cbd4-4bef-92ae-fed607cc2bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:42.746904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:42.747948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:42.748869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:42.748869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:42.748869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.748869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:42.748869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 319/625] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:42.749640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99536-aceb-43e6-b115-8cd3f76abcdc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.750006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99536-aceb-43e6-b115-8cd3f76abcdc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.750298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99536-aceb-43e6-b115-8cd3f76abcdc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.756151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 320/626] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:42.756151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f054ff5-b275-4e74-9234-3e7b54613480 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.756151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f054ff5-b275-4e74-9234-3e7b54613480 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.756151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f054ff5-b275-4e74-9234-3e7b54613480 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.757910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.761025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 321/627] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:42.762276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851e6f-dde1-4ae1-8ceb-d0443a7f6dca None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.762719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851e6f-dde1-4ae1-8ceb-d0443a7f6dca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.762991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851e6f-dde1-4ae1-8ceb-d0443a7f6dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.767040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.768513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.770381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.770381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.782971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-40851e6f-dde1-4ae1-8ceb-d0443a7f6dca None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:42.784248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.784461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-40851e6f-dde1-4ae1-8ceb-d0443a7f6dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:42.785676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:42.807143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:42.809024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 307/628] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:42.811856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.852435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.883928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 322/629] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:42.885349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.885703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.886033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.907706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:42.908984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:42.971214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbf3eb-8677-42b9-8d8d-0a21414bbd6c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.001511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 323/630] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:43.001511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.028266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.030020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.030523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.031013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.032442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:43.033067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.035762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.057995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 308/631] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:43.059599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.131972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.144963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b3887-01d9-4a90-aa96-f6a20192a754 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.156360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.189924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.190831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.191632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.192098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.193580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:43.194120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.195941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.220589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 324/632] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:43.220589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.220976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b912-3d41-4ffb-b25e-a3ba92dc4f2c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.261533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.262474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.262819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.263201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.277945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:43.286127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.287933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 309/633] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:43.290159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.370839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd5949-642d-497d-b8c8-a8fd4b07d801 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.428804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.429938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.430401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.430870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.434907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36b5a-cd16-4181-afb1-0f791863da34 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.438803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.449366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:43.460781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.462913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 325/634] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:43.466105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.490314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.491520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.492125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.492506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.502206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.503780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.504633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:43.521801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.523062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:43.524150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:43.526470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 310/635] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:43.527558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.632329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c207da-014e-4e0c-acc0-56bdd473c299 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.698380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb223b-1da5-46af-aba1-5960d406225b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.704400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.705996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.706554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.707062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.720796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.723724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:43.749004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.750830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 326/636] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:43.760391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.761568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.761568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.761568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.761568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.762044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:43.762044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.762867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.786192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 311/637] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:43.786192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.819054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.936713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.944264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273dc1-585a-45ff-9652-ed09fda82d5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.955629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09865842-c9da-40b6-8a4c-d3a8d2afeb37 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.002263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.002970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.003334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.003522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.004335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.004918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.006118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.018350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.019576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.024903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 312/638] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.025092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.025092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.025254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.025340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.025340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.025477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.042505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 327/639] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.044114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.130559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.151054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2dfd-b666-4d34-8a93-9f21623bf91e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.193713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.194382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.194695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.194964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.195762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.196076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.197562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.212243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 313/640] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.213016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.250736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52b169-0c0a-4f4e-b9e5-fe90eaa82171 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.292598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.293397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.293674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.293945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.305693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.307198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:44.326922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:44.328856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 328/641] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:44.331004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.336178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.363382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d5a7-7e7e-4a2f-913d-983931af1fb9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.413936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.415008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.415450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.417241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.418559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=5e2880530a3d43b482fb27ac23701293) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.419058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.422096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.431478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 314/642] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/role_assignments?user.id=5e2880530a3d43b482fb27ac23701293&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.432634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.450208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.502888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab987747-be82-40ce-9a6f-0859ef1dc8af None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.564468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.565374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.565660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.565937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.580823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.582810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0415b7ab-5013-4f5d-b0c3-d1fe140119c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:44.604622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:44.606691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 329/643] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb73f-04ae-4b83-b04c-88ba04ab2814 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb73f-04ae-4b83-b04c-88ba04ab2814 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.609755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb73f-04ae-4b83-b04c-88ba04ab2814 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.615066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 330/644] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:44.617589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb73f-04ae-4b83-b04c-88ba04ab2814 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.631809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2b94-030a-4adb-9bfb-fd0a03c6a78c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.699997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.701637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.702138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.703582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/71aecafdbfad40d4979f6f6358a58109/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.726809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=71aecafdbfad40d4979f6f6358a58109, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.728362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.750473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 315/645] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:44 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/71aecafdbfad40d4979f6f6358a58109/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:44.752275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.761113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.882878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.886117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb73f-04ae-4b83-b04c-88ba04ab2814 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.960034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.960735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.961030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.961346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.962066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.962377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.965108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f81785-1c5d-4cb5-932a-cf8f1c5147cd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.981558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 331/646] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.984320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.984676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.984880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.990890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c648d-97e9-4f8a-ad94-9e0896fa4f81 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.014542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:45.016913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:45.059069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.060584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.061382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.062166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.063831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.064646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.067788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.095444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 316/647] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:45.101788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.182952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 332/648] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:45.189077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.290018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.337349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101255c9-a1e6-4903-b36a-9c93de32ab2a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.367022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.379186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.380005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.380340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.380610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.391307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.392815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.401180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83c97-aecd-4f9b-b360-8f325443726e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:45.413017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.414940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 317/649] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.418829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.469703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.471146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.471644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.472127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.486856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.489656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:45.520645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.524308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 333/650] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.535141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.539409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a58b80-7532-48b0-925a-042a589f4503 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.615036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.617083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.618168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.619044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.642356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:45.660025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.661890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:45.663611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:45.668477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 318/651] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.676689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.716490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef473ff0-8e34-4bb1-b46c-6db5d7ce698b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.721334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.770976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.791686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.792908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.793404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.793895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.795746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.796218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.799137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.837186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 334/652] 10.222.0.77 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:45.838890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.902346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803f24f-948a-45fe-9d8a-52c05e40cc12 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.974220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.976172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.977044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.977828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.979859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.980727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.984646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.030491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 319/653] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:46.032316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7031698-3c7c-4b38-ac35-b276800a1b3a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.032990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.107854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.109050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.109554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.110066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.111620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:46.112190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.114792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.138489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 335/654] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:46.138489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.193986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.238557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.251968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb50f-bce2-427f-8b4e-c99745fb916d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.323166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.324922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.325725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.326619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.355216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78408-b103-469b-b877-ab112534aa36 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.425688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.427150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.427351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.427845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.465552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.468912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.580359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.583350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.634114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.663306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.678453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 320/655] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:46.680391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.823687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 336/656] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:46.823687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.891523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c50e0-2ead-4293-83d3-72f8b7a02b56 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.973169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.975512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.975998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.976489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.005111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.009273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.034107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f70895-ca2f-4952-a2c0-0bc9b78888df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:47.034634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.043239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 321/657] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.045543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.078711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.090991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.092434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.092718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.092994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.104235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.105717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.115832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-53b93936-28b0-481c-b448-8580af9e5da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:47.127962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.129183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.130309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:47.130309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:47.130309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.130309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:47.130309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 337/658] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.131770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45949-f6f8-4ab7-a920-c20a5aaeb62a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.132263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45949-f6f8-4ab7-a920-c20a5aaeb62a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.132762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45949-f6f8-4ab7-a920-c20a5aaeb62a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.135672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 338/659] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:47.136726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e45949-f6f8-4ab7-a920-c20a5aaeb62a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.190925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d27a-9bd5-4609-a8ff-3963051c2fc8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.236498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.237867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.238365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.238883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.255703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.257159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.261284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e45949-f6f8-4ab7-a920-c20a5aaeb62a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1b271b9b-8737-4b7b-bf25-46c23bb8b779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:47.277250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.279713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 322/660] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17d2b2-9d83-4365-9dbf-d2b5b6a24c86 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17d2b2-9d83-4365-9dbf-d2b5b6a24c86 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.281404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17d2b2-9d83-4365-9dbf-d2b5b6a24c86 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.282998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 323/661] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:47.283770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17d2b2-9d83-4365-9dbf-d2b5b6a24c86 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.321158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.322254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.322763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.323205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.324489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.325019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.327826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2e450-35a6-4a8b-95c1-9dcccebaac1d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.349994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 339/662] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.352388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.352888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.353408 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.380729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:47.382854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:47.423826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.428450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.430288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17d2b2-9d83-4365-9dbf-d2b5b6a24c86 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.476191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.477184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.477402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.477750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.478656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 340/663] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:47.478850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.479190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.479517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.481019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb6f9-624f-470e-b051-3046d240394f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.500384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 324/664] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.506342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.506727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.507152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.529880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:47.531353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:47.609623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 325/665] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:47.610431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.669511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db14f7-2735-4c0b-b9cf-cae627240b87 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.724238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.725259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.725717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.726174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.727066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.727280 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.727679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.729841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.744133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4a81-e307-4e9c-90e8-07b2814988be None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.754822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 341/666] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.754822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.799933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.804084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.805348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.805808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.806239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.819003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.821607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:47.848144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.849880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:47.851740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 326/667] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.853138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.919043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddc3af-bd99-48b3-9ecf-627a33d86f96 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.965540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.967058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.967449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.967880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.975728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.977251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.991675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33caa-23a9-462a-8cf3-9d40dcbfefea None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:47.993895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.995191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 342/668] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.997707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.033550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.034360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.034753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.035025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.047039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.048637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:48.068512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.070283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 327/669] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.072354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.075349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.134288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.153541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f389d6-bce0-4033-bb42-752a3116aa42 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.213583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.214812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.215265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.215724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.227630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2eb0c-9e5b-4eaf-b0cd-2124ce6d73ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.234126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:48.248044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.249227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:48.250500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:48.251425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 343/670] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.252085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.288323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.289639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.290112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.290572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.298623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.300116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:48.315297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.316364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:48.317446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 328/671] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.317959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.394266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675d03-8513-4471-865e-951f8dd7fb96 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.423522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.434009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.434737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.435022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.435337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.436207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.436515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.438293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.464319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 344/672] 10.222.0.77 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.465120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.471951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.484013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2b507-0c55-4b29-bb1b-f7456db64ecd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.525030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.525709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.525989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.526255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.527111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.527416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.528944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.547227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 329/673] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.548252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.618042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05370eb-fb62-4ba7-8e68-2aea3ddd547e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.675430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.676329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.676706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.677169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.678025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.678436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.679945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.694762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 345/674] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.695911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.696606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273921-30d2-43f7-861f-7b6a0cbbc14e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.739631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.755632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.756311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.756589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.756859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.757632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.757931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.759406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.769074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 330/675] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.769936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.793739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.843175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e674-daae-4a96-b0bb-0829010d8cbb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.887869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.888686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.889002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.889289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.890139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.890488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.891999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.906176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 346/676] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.907098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.920819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daf911-2f3f-4a9d-8234-9e172218f886 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.978353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.979531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.980041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.980550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.029167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6031f9-921c-45ed-831d-b83f32a20cbb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.047970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.068191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.069704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.075043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.094147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.095240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.095700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.096173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.097779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=527930459ed340c0b09ee6c6da770a21) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.098324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.101138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.113633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 347/677] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/role_assignments?user.id=527930459ed340c0b09ee6c6da770a21&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.114438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.204344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 331/678] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:49.206620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.316155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8841044-db6a-456a-9a0f-2fb036a690b5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.365541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.366643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.367074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.367557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.368847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.371358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.376400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.406494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 348/679] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:49 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:49.408109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.409159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f50bd-b2ce-419c-ad7f-82e98e6032e4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.479348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.480654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.481360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.481814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.509403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.510326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.512389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:49.554061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:49.556097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:49.558156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:49.558156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:49.558156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:49.558156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:49.563029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 332/680] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:49.564564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.632782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9780271-b060-47cc-9994-30f90f510097 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.701996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.703517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.704252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.705205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.707045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.707854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.710915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.723607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 349/681] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.725023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.789026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35686-225a-4bd0-8ca8-590219cd769d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.926256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.936102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.936573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.937050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.955678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.004935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.006481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:50.094111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:50.096086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:50.098024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:50.098024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:50.098024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:50.098024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:50.115131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 333/682] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 551 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:50.123594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.182303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab078b-b646-46d2-8e15-845ede52580b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.210231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.295522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.297151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.298039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.298819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.301217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.302178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.305133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.331035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 350/683] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 606 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.336144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880afd43-3d3f-42ca-9b82-7bc94091b4f1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.344077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.388567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.389541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.389857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.390147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.390930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.391241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.392680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.406475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 334/684] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.415731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.482655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.551748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701424b1-eaa2-4954-b4ea-e9a732328c8d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.582601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65b89d-c62f-4904-951b-07949b425d4b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.626367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.627943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.628541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.629050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.652548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.655804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.656722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.657555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.657838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.658115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.670928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.672508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:50.693039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:50.696678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:50.704949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 351/685] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:50.707471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:50.709689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:50.712921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:50.716254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 335/686] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:50.720788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.781619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.941748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74549ee2-87ee-4360-9a86-da1d51d8683b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.955826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992063a-d392-4b1f-9bb4-2d3cc9491b45 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.976833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.034520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.035741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.036224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.036667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.036793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.037796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.038304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.038443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.038824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.038993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.040353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.040888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.041619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.047856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.092770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 336/687] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:51.097117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 352/688] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:51.105430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.110469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.287985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.311717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39119969-43eb-4cbf-85b5-ef1af627b3ab None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.378922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f70970-6630-48a7-af49-8ef803be68f5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.388137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.389944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.392697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.393590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.395689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=1747a2eb4850417d8d02a2b0a8d0cff7) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.404691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.407417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.422558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 353/689] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/role_assignments?user.id=1747a2eb4850417d8d02a2b0a8d0cff7&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:51.512754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.514055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.514590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.515100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.615096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.709571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.712708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.923016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 337/690] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:24:51.930217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.178627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f6e0a-c36e-4bbb-8337-2b0f38bdee3b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.246360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.248366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.248989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.249515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.274806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.277790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:52.312284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:52.314766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:52.316491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:52.316491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:52.316491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:52.316491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:52.316491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 354/691] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:52.332293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.509415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:52.598468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd45f3-03e5-4b70-b77d-8cd18a848e60 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.688351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.689539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.689953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.690468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.691912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:52.692456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.695293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:52.720944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 338/692] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:52.732700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.741305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.020205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34387a-9849-4256-97bf-80bb2675d8bb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.099333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.100184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.100490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.100759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.109426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.110988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a1adf542-2a5e-442e-9e28-4b7c7ab81e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:53.132213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:53.133936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:53.135708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 355/693] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:53.152279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.236857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17f1-f471-4456-9e5e-a524c6364aef None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.237519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17f1-f471-4456-9e5e-a524c6364aef None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.238010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17f1-f471-4456-9e5e-a524c6364aef None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.240116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 356/694] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:53.246115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a32d93-8c42-497e-8bf0-44db7b0ff2f9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.246831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a32d93-8c42-497e-8bf0-44db7b0ff2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.247339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a32d93-8c42-497e-8bf0-44db7b0ff2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.299472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-58a32d93-8c42-497e-8bf0-44db7b0ff2f9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:53.301517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58a32d93-8c42-497e-8bf0-44db7b0ff2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:53.410116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d723a-0138-4bb8-81da-a031b4104754 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.445220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.467003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 357/695] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:53.467917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6104-2562-429a-8e9f-25343113dbaa None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.468421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6104-2562-429a-8e9f-25343113dbaa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.468916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6104-2562-429a-8e9f-25343113dbaa None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.471830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 358/696] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:53.472923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.474585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8803bfe-ba0d-41ba-9ba3-e46610be35d4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.475316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8803bfe-ba0d-41ba-9ba3-e46610be35d4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.476139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8803bfe-ba0d-41ba-9ba3-e46610be35d4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.485822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 359/697] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:53.488166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.488810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.489387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.490366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.491661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.492448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.493173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.494948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.495885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.499448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73cf23-f112-4463-8699-900cb506e70d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.517620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:53.519796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:53.524294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 339/698] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.526890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.527408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.527964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.561760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:53.563789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:53.670446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 360/699] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:53.670446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.702982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 340/700] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:53.704156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.836020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6deb39-3793-4a43-aa5c-7488c727cef7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.859539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.876607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecdfcc-3a4e-4965-9cbf-87d3a15a947b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.895738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.896892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.897392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.897867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.899628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=71aecafdbfad40d4979f6f6358a58109) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.900470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.903546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.908051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.917043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 341/701] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/role_assignments?user.id=71aecafdbfad40d4979f6f6358a58109&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.919812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.928258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.929980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.930717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.931888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.961989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:53.983031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:53.984865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:53.986551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 361/702] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:53.989268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.169882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cb6ef-fd15-4920-a595-80d1d6b0ce3e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.182211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96268656-f659-46d9-b982-9c47a7870b15 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.237123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.237958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.238254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.238576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.250769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.252179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.252684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.253183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.253343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:54.254620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.257386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:54.261239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:54.263044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 362/703] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:54.265965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:54.293116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:54.295371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:54.297346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 342/704] 10.222.0.77 () {56 vars in 1129 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:54.299938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.357222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.370245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.456560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b5d9f-1a60-4ad3-91e4-fe4d5a6867b1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.459386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77611e9-2f8f-4192-ab3d-f95484eda6c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.510841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.512088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.512730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.513329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.514859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.515650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.518008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.534869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.534869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.534869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.534869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.535268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.535589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 363/705] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:54.535757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.536956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.538536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.556419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 343/706] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:54.558496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.724591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d556-0128-4078-aaf6-c04ab840628c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.728656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96f3df-49fa-4c28-a6c0-41a18b3898fe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.734671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.739772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.780166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.782093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.782706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.783466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.799245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.801214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.801855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.803174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.803764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.804351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.806010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.806692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.809619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.824671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 364/707] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:54.824671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:54.831617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:54.832614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:54.833552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:54.833552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:54.833552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:54.833552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:54.836097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 344/708] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:54.842781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.984864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1a14-fd3d-4699-84b2-37772c38d1c3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.027839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85965e-ea2b-41a7-ac6e-d128b1beb14e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.032358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.034542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.035068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.035554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.049406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.051713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:55.075515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.076559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.077523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:55.077523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:55.077523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.077523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:55.080477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 365/709] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:55.081738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.100008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.100971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.101437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.101862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.103340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.103838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.105875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.118765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 345/710] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:55.120049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.120882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.172251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.266485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875eab-d028-4d43-880d-bd36ab3027a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.276017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4487e-e213-4544-aa5b-e640105cb04b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.320156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.321250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.321750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.322237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.323387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.323941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.325699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.342395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 346/711] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:55.348658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.350457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.351784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.352333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.352839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.355252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.355252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.364551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.407997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 366/712] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:55.415991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.467936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.496739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e2dea-7e7a-4092-82d0-1f19260ef8c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.564956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.566928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.567361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.567765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.585746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.587360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.590111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-73386386-3729-4a7f-b1be-ea72269219c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:55.624008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.625442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:55.626447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:55.630534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 347/713] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:55.632063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5062-f990-4028-b350-01aaa1cbabdb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.633734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04050f-b6c9-491d-88dd-074d3208d18b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.634518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04050f-b6c9-491d-88dd-074d3208d18b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.635504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04050f-b6c9-491d-88dd-074d3208d18b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.645285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 348/714] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:55.647341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04050f-b6c9-491d-88dd-074d3208d18b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.706367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.707779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.708261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.708836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.724296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.726940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5f373984-8584-42e4-aa9c-87f577919597 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:55.757893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:55.762263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 367/715] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:55.764002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.764002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.764728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.794516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:55.796638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:55.857807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04050f-b6c9-491d-88dd-074d3208d18b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.921711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.923475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.924280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.925053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.926911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 368/716] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:55.927419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.928188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.928433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.931510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.981613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 349/717] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:55.983219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.988202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.091696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.142394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45d26f-cf31-4eb1-a21c-98614b97bc70 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.193677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17bc6-23f3-4543-9056-5697dc3c4d95 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.214618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.215966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.216661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.217434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.219387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:56.220089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.223335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.260874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 369/718] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:56.262823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.264108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.265851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.266037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.266541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.267899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.270860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:56.295495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:56.297443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:56.299108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 350/719] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/services/network => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:56.305070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.461025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.473862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc105-092a-46ce-a762-c641b928d310 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.531679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.546245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958411-b1fa-4fbe-92f9-7c10405191c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.547102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.548465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.549102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.549673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.619172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.620928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.621681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.623384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.697039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.699965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.774931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.778447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.920004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 370/720] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:56.921738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.923055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.986782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.999191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 351/721] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:57.001611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.153626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fef21-7bf2-409c-bcc9-cd10490f41df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.212766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8ad2-050c-4de6-87a9-6936915ef437 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.235784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.237568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.238435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.239281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.241399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] RBAC: Adding query filter params (name=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.242378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.245570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.262632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 371/722] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.265094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.284094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.285487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.285989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.286526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.305370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.306936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5286ecb2-38ee-46ff-97b3-3f44b74caa2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:57.328754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.330761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 352/723] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd979e-d01d-4e97-8e66-6e98e556a221 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.332640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd979e-d01d-4e97-8e66-6e98e556a221 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.333402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd979e-d01d-4e97-8e66-6e98e556a221 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.338481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 353/724] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:57.339475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd979e-d01d-4e97-8e66-6e98e556a221 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.362228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.410795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.422062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da99b9-8622-42cb-9e4c-045bb1fbaa74 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.477912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.479242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.479721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.480197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.498810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.500375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd979e-d01d-4e97-8e66-6e98e556a221 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.501459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:57.534021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.537521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 372/725] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.539507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.575186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.577103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.577810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.578497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.579515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] RBAC: Adding query filter params (type=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.580258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.582731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23afc8e-5eba-4528-bf5a-34b2b69d9a3a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.598483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 354/726] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.598483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.598483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.598483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.629222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:57.632419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:57.758083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e49b7-819c-43ad-a631-356e24652f9a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.777635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 355/727] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:57.779289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.811997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.831662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.832968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.833564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.834158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.835809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.836521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.839472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.864547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 373/728] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.865987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.874798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.004286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d108d-53e7-4b1d-aa03-e0361294a6ec None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.053960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3fc55-f193-4943-8a7b-862226d9f741 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.063185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.064271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.064756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.065240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.066709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.067214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.069871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.096562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 356/729] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.096562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.123654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.125024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.125522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.126011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.142722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-88f32ac3-6f35-4280-98d6-3480baa19df0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:58.153269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:58.154861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:58.155757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:58.157164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 374/730] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:58.159538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02d0d-161f-43bf-b291-533ef695d7c7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.160071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02d0d-161f-43bf-b291-533ef695d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.160557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff02d0d-161f-43bf-b291-533ef695d7c7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.163657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 375/731] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:24:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:58.165241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02d0d-161f-43bf-b291-533ef695d7c7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.237958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.270765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.298855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2a102-6f4e-4454-83f7-3ea0dfb6e282 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.327179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff02d0d-161f-43bf-b291-533ef695d7c7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.360355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.361789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.362261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.362794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.373835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.374959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.375420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.375859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.384965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.386770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.388066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.390832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:58.405147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:58.406986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:58.408773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 376/732] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:58.409750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d17b06ee-633f-4ed9-836d-0940b3f73874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:58.420707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:58.424054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 357/733] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.427918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.430092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.459037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:58.461293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:58.566712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0fd97-7d23-462a-90d3-db72d213d40d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.600390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 358/734] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:58.601695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.616553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.637468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.638703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.639191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.639725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.641071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.641602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.644260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.670455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 377/735] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.670455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.717755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.773298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042c1f-ca03-486a-8f27-f9cf4948b0d3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.820926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.821876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.822346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.822867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.823899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.824397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.826353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.842909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 359/736] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.844057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.869704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9e3b-b7c6-4543-88a9-33bc3b2a9e8b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.916235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.916926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.917198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.917476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.918296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.918644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.920707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.936228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 378/737] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.936879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.006700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.021547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581fe27-2a40-4f9a-be83-2e27bb05eb71 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.062696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.083287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.084349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.084820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.085277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.086578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.088883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.105389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb069b8-2472-4c9a-8a6e-6fdc3e34ed48 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.114354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 360/738] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:24:58 2022] POST /identity/v3/services => generated 251 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:59.116215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.207919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.209372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.209858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.210330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.234868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.237822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:59.274807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:59.276559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:59.278245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:59.278245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:59.278245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:59.278245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:24:59.283141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 379/739] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:59.284878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.332925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dacc4d-6e0b-4477-97d4-61d27403cfd2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.390368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.391994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.392663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.393360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.395176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.395987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.398671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.409494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 361/740] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/role_assignments?user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:59.411324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.507579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.529380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.557854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2a00c-c4e3-405c-9dd9-eb3057d7f85e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.592242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbc1c0-a1e1-4f8f-944e-18104727ac4f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.651479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.654505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.654505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.654505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.654505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.654505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.655878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.675500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.677068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.677803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.678545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.680362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.681172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.682960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 362/741] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:59.684692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.691475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.702596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 380/742] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:59.916081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1997-5c5d-4ff2-830f-372a5c25e269 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.931022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.977999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.978864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.979147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.979426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.989146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.990718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:00.015371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:00.016400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:00.017909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 363/743] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:00.024748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.173899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302033-d923-483d-91eb-ad48887ab5eb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.214893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.215953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.216960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.217498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.218709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:00.219261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.220925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2416d-24fb-4cbf-81ab-06af63506a67 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.243064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 381/744] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:00.251951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.510334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35239acb-2da2-4687-8787-9e601419fc57 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.559990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.561661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.562165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.562681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.624155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.693094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.695057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.784013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eac6a-ce70-41cc-a940-4ce1b514ca12 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.784435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eac6a-ce70-41cc-a940-4ce1b514ca12 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.784940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eac6a-ce70-41cc-a940-4ce1b514ca12 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.796404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 382/745] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:00.808943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.809447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.810890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.839030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 364/746] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:25:00.847562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.857986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:00.860204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:00.889969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.958679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.002651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579813-f229-4824-83a6-d1bfe3ea15df None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.048753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.049471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.049748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.050014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.063699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 383/747] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:25:01.068093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.069576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:01.091561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:01.093298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:01.095113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:01.095113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:01.095113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:01.095113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:01.095113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 365/748] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:01.103471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.113812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.296962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce736b-1e60-4f1b-a5b5-6540f0c3833b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.340383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.340983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.341884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.342451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.342960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:01.343265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.345402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.361629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 384/749] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:01.365868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.377629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.488622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106357ad-9f87-462d-8e18-31e4292a7e2f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.498237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971993a4-e1e2-47c4-91cd-c68bd618a792 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.530012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.531632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.533185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.533888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.543394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.545731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:01.564172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:01.565140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:01.566032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:01.569505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 385/750] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:01.579758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.582826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.647820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.649152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.649936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.650504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.652107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.655491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.667041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.703766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c37f-c029-4b0b-b19d-8845b6fb8103 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.710498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 366/751] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:01 2022] POST /identity/v3/services => generated 240 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:01.768037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.769050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.769531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.769984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.771125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:01.771618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.773275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa5054-9c50-44cb-bcd1-a9498e04ee0a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.791123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 386/752] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:01.800416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.972797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ed95-c65a-4459-81b7-73a4e120f139 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.023865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.024544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.024871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.025150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.026007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=71aecafdbfad40d4979f6f6358a58109) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:02.026319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.027915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f02d5-b341-4c71-a3c1-c61b9ec9f72f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.039029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 367/753] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/role_assignments?user.id=71aecafdbfad40d4979f6f6358a58109&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:02.288626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6596-3c81-48a0-ab4c-8acf7991dfde None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.289136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6596-3c81-48a0-ab4c-8acf7991dfde None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.289618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6596-3c81-48a0-ab4c-8acf7991dfde None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.293038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 387/754] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:02.299522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff79a5-02bf-4f0b-8b60-0e69b32dda5a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.299811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff79a5-02bf-4f0b-8b60-0e69b32dda5a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.300092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff79a5-02bf-4f0b-8b60-0e69b32dda5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.321111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-08ff79a5-02bf-4f0b-8b60-0e69b32dda5a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:02.322352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-08ff79a5-02bf-4f0b-8b60-0e69b32dda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:02.377274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.424913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 368/755] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:02.468777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6596-3c81-48a0-ab4c-8acf7991dfde None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.734984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957bb0e-69b2-4fbe-911f-5f36ee102d13 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.736026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957bb0e-69b2-4fbe-911f-5f36ee102d13 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.736500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957bb0e-69b2-4fbe-911f-5f36ee102d13 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.739787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 369/756] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:02.746188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.746533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.746804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.749880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6596-3c81-48a0-ab4c-8acf7991dfde None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.785318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:02.787435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:02.813821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.834716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.835568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.835856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.836138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.862302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:02.878602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:02.879630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:02.880565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:02.882441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 388/757] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:02.892124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.974437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 370/758] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:03.029537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574883-4da0-427e-ac39-9099f58016ea None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.043267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.077214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.079393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.080376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.081843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.083116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.085052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.087911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.111985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 389/759] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.116712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.207002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.265210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b829f-9ded-4774-b92b-1ddd9259999a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.268051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.316481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.317636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.317914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.318186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.318899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.320309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.320669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8a99b-3015-4722-860c-57278b24738a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-55561b2e-d42b-45a7-b219-2a2758161ac4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.336437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.337464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:03.338791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 371/760] 10.222.0.77 () {56 vars in 1129 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.339665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325effc-dcba-4266-b38a-3d694eedea1f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.340077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325effc-dcba-4266-b38a-3d694eedea1f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.340239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325effc-dcba-4266-b38a-3d694eedea1f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.345813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 372/761] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:03.347697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325effc-dcba-4266-b38a-3d694eedea1f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.397998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.399611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.400111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.400601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.423549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.426786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-30d5a76e-2a10-45f3-8fb4-99a690f7dd36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:03.466446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.468596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.470808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:03.470808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:03.470808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.470808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:03.470808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 390/762] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.472440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.472440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.476254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.480643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325effc-dcba-4266-b38a-3d694eedea1f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.515798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:03.519331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:03.528549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.529233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.529520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.529796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.530567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.530877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.532400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.539823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 373/763] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.540745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.623094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.634456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 391/764] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:03.635374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.712184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.764328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4d8b9-3b0a-4af7-8a1e-6fec63574fe2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.837834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ea6e-2aa9-4850-bd9e-53d7da8bb81c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.844729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.845865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.846354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.846863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.849208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.849752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.851906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.885731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 374/765] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.886999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.887919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.888208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.888485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.889039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.889223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.889539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.891166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.899328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 392/766] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.900250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.059841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.077557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9aa7-5235-4723-a5d4-7cb7c3b5d717 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.079018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.093123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e001432-8c43-46f4-8477-d7d2136bbbab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.147050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.147875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.148163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.148437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.155413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.157272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.158542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.159191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.159693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.160678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:04.178631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:04.180500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 393/767] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:04.182354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.192450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:04.218636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:04.220458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 375/768] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:04.222093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.348040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eb0eb-a787-4b13-a7cc-ca5f29be156e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.388737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.389436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.389724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.389991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.390904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.391220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.392710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.394394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326745d-686d-45b8-850b-1f452e1e79f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.414274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 376/769] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.415601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.428875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.468378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.469977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.470918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.471625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.473735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] RBAC: Adding query filter params (interface=public, service_id=425ad437912d4424a5ba1ecc0aa75103, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.474609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.477837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.495618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 394/770] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/endpoints?service_id=425ad437912d4424a5ba1ecc0aa75103&interface=public®ion_id=RegionOne => generated 194 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.498179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.531915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.543971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad27652-7823-4166-9ed1-26d56e742e65 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.587768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.588658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.589063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.589440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.590350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.590793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.592364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.601872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 377/771] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.602640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.693198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe911f-5e8e-4787-8c2e-3a936de70e42 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.695489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.739806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa5f6-5763-4b93-b7b6-76669d4fd1ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.742475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.743413 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.743705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.743986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.757532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.759225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2011b77e-73f0-42dc-bbed-b9143f3c9cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:04.779814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:04.781437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 395/772] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65f34-2e41-40c3-8651-9cb01745c730 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65f34-2e41-40c3-8651-9cb01745c730 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.783894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65f34-2e41-40c3-8651-9cb01745c730 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.790841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 396/773] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:04.790841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65f34-2e41-40c3-8651-9cb01745c730 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.797164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.798205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.798641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.799030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.799879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.801863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.809421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 378/774] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.810757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.939540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.015661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65f34-2e41-40c3-8651-9cb01745c730 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.016725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47b2c-4a4f-4588-9b59-ba0bd9638eca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.070243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.072801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.073584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.074375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.079998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.081602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.081804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.082124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.083562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:05.084081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.086566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3755c4b-4f70-497e-a539-69853305a26c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.100652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.104446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.110053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 379/775] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:05.117489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0269-debc-41fa-bcd2-28e0326d843d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.117489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0269-debc-41fa-bcd2-28e0326d843d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.117489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0269-debc-41fa-bcd2-28e0326d843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.133717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.144225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7e3a0269-debc-41fa-bcd2-28e0326d843d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:05.146281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e3a0269-debc-41fa-bcd2-28e0326d843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:05.149415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.150344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.151323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:05.151323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:05.151323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:05.151323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:05.152746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 397/776] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.153961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.258765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 380/777] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:05.261674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6904f3-c3a9-4f01-acb1-870f846bdde8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.262181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6904f3-c3a9-4f01-acb1-870f846bdde8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.262713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6904f3-c3a9-4f01-acb1-870f846bdde8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.270223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 381/778] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:05.270223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6904f3-c3a9-4f01-acb1-870f846bdde8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.292291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8481db-8007-4e4c-9f3f-1fa3e2004afc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.315045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.337973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.338919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.339175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.339415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.350045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.351613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b9eb203f-eb04-4b16-b283-a33486cac021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:05.368592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.369870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 398/779] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.378093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.380565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.397906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:05.399513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:05.437822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6904f3-c3a9-4f01-acb1-870f846bdde8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.472321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.487550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 399/780] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:05.487762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.488540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.488941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.489166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.489398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.490247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:05.490687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.492200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.517168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 382/781] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:05.518287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.642973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2ca13-e146-4b45-94f8-0005cc82262d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.655750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.670823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571a12-92bd-4084-97ad-06b6c4c458e2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.708293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.709139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.709459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.709734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.716983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.718096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.718623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.719106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.722295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:05.727669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:05.728254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:05.730608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.732125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:05.733018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:05.733686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.733763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 400/782] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.733763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.752741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 383/783] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:05.753608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.818453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.875226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605983d-f566-4220-bf85-9a2272acce60 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.887517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cedd1-bf53-4797-b2bc-7d3867b071a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.914946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.915782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.916063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.916390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.926232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.927867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:05.945401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.946486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:05.947451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:05.947996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.947996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.948675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.948777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.950013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.950790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 401/784] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.952018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.952954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.977010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.978086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.978986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:05.979725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 384/785] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.980996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.990324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.093084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9bd5-ede8-489e-adda-63dc119a9a92 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.158788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.159490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.159771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.160051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.160906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.161183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.162637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.172934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 402/786] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.173894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.180555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ac96e-5690-455e-a59e-e1c10bbbde90 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.200850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.246036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.246621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.246914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.247214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.249350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=bd1cd1cd83fa43cab0dcbd1698c715fd, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.250891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.270935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 385/787] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:05 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:06.272855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.323069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.338268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568cbed-bab6-4004-aef6-0e6a4803ff68 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.379681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.380734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.381180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.381629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.382763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.383291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.384992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.410874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 403/788] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.412716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.495524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7bab-46e5-40e6-ad50-7e28078dfe86 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.537671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639314bd-3573-4fda-99c4-a78752b3b79e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.577707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.579169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.580012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.580604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.582397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.583072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.585582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.586991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.588044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.588529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.588971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.598318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 386/789] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.602444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.612200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.614176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.624815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:06.636305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:06.637241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:06.638761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:06.638761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:06.638761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.638761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:06.641411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 404/790] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:06.642593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.681893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.795715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190c075-5701-44bb-9ff7-45c963813d90 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.813654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1d76-410d-4b8f-b381-1d2ea47d78af None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.864501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.866216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.866721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.867177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.867569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.868830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.869415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.869961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.871475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.873037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.875803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.883494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.885357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.891473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 405/791] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.891473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:06.914754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:06.915747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:06.916663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:06.916663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:06.916663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.916663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:06.916663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 387/792] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:06.917328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.028471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.030826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.050579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb47c-cd3f-4c17-b250-cf4483f7de50 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.094338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.095339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.095897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.096336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.097338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] RBAC: Adding query filter params (interface=public, service_id=80c83e4762a14fd9b73bbffc138c89af, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.097861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.099498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.108818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 388/793] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/endpoints?service_id=80c83e4762a14fd9b73bbffc138c89af&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.109436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d69cd-54e1-4fef-90af-2111642c6e8a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.110211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.179181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.180265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.180752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.181511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.182527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.183611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.186155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.203802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 406/794] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.205227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.278851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35892563-f7b1-4780-aab7-93a72cad5dc1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.319363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.320412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.320891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.321350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.322290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.322814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.324465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.343536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 389/795] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.344834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.357608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.412933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e784f55-43e4-4aa0-8866-c238fb505c9a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.464707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc00f-743c-40d6-a56c-54e1562ee41b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.467994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.492742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.494296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.494859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.495380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.496726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.500074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.507684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.508500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.508781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.509108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.514608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 407/796] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/services => generated 772 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.514608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.517966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.519552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e98891f1-eb91-4f6b-ac09-565cbbc3fe23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:07.537121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.539386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 390/797] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1285a-a3e9-4770-9a36-8c6184705604 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.541444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1285a-a3e9-4770-9a36-8c6184705604 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.543810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1285a-a3e9-4770-9a36-8c6184705604 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.543810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 391/798] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:07.545528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1285a-a3e9-4770-9a36-8c6184705604 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.668532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.712572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1285a-a3e9-4770-9a36-8c6184705604 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.735485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455919b-fb00-4489-b683-0b6d56f55d9c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.756034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.757274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.757274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.757588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.758296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.758645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.760175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42944007-11b5-47f4-9dda-4198f144b3ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.782191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 392/799] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.784278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.784278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.784644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.814320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.815202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.815487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.815795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.819007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:07.821053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:07.825021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.826654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:07.845299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.847350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 408/800] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:07.849725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.923757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.938725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 393/801] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:07.939743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.000418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb91694-45d5-4bb6-abce-ef8e91d655e5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.010211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.046224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.047387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.047682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.047960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.064816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.067296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.073002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa7360-41c1-4faf-9f15-19e5a74bd16d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:08.093556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.094605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.095591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:08.095591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:08.095591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.095591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:08.095591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 409/802] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:08.096088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.134370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.135473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.135760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.136039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.136876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.137191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.139534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.162451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 394/803] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.162451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.228037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270506-0777-44cd-b3ef-fdfa31652514 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.231065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.279636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.281602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.282071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.282526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.283367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.285708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.300484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47d4-67be-4acf-a32a-55ca97c48c47 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.306839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.308052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:08.309128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:08.312808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 410/804] 10.222.0.77 () {56 vars in 1129 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:08.313992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.328390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.358338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.359605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.359992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.360370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.361227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.361651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.363387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.386869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 395/805] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.388354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.484091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acedc7-f5f7-4aec-92a8-550e4a2dcbfb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.526773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.528006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.528946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.529346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/e98cf2fa9bee4dcfa9f7410d19b67f7e/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.548632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=e98cf2fa9bee4dcfa9f7410d19b67f7e, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.550227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.567355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 411/806] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:08 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/e98cf2fa9bee4dcfa9f7410d19b67f7e/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:08.568551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.603010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.633465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8a3f3-daea-431f-aa04-3bcf92f3c087 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.739030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.739886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.740174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.740447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.741191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.741501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.742971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.758286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 396/807] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.767350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.790341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3f50-9804-4c2d-9ba4-e586d238e52d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.829678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.843689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.844948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.844948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.845132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.853269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.854790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:08.872432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:08.874331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:08.878683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 412/808] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:08.881338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.906819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9381b-0eec-4b17-9943-eb7035f60621 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.962009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.963580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.963910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.964183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.964871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.965183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.967350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.986482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 397/809] 10.222.0.77 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:08.992417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.125397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.178672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ceb12-b52d-423b-8abd-bd9e51959c55 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.212815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00ff18-c4a3-424d-b635-59c7f1cda727 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.220757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.242896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.243596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.243883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.244164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.244989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.245302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.246894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e330e-e7e8-4ec7-a2e1-ebee3c4e4a4b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.261250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 413/810] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:09.262752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7387-d354-433a-8194-7bc48c02af12 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.263045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7387-d354-433a-8194-7bc48c02af12 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.263325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7387-d354-433a-8194-7bc48c02af12 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.269090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 414/811] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:09.270679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.270975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.271260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.290557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:09.291788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:09.298085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.299093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.299615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.300007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.301397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.304069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.356034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 398/812] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:09.359490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.427382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 415/813] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:09.468110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.599455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.606307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35886-303e-4288-bf2d-2560f77e209b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.657617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04effa4-7657-4b0b-8e88-884e637d584e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.707661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.709208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.709496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.709776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/71aecafdbfad40d4979f6f6358a58109/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.711621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.712059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=71aecafdbfad40d4979f6f6358a58109, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.713556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.713967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.714866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.715628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.740553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:09.743829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 399/814] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:09 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/71aecafdbfad40d4979f6f6358a58109/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:09.753590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:09.755426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:09.757104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:09.764137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 416/815] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:09.771391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.830070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.985513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68321f9-8a77-49dc-8e9f-52c0c57cce76 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.042983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.043684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.044022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.044300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.045082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:10.045390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.046839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.056894 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 400/816] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:10.063516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.187627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aab27-4424-4d68-93da-86ba63e03138 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.212467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.237869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.238645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.238937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.239219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.252357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.253866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b81aca6a-2526-4f1f-a221-01986ea3b040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:10.280951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:10.281914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:10.282870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:10.282870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:10.282870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:10.282870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:10.284563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 417/817] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:10.289607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.396289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.429864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b716b2-ae78-4961-bd27-d4697cf54573 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.457460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81178a-e0fd-4b5a-bdac-7e43e494cd96 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.457791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81178a-e0fd-4b5a-bdac-7e43e494cd96 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.458013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81178a-e0fd-4b5a-bdac-7e43e494cd96 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.459771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 418/818] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:10.467338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.467338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.467338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.487831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:10.489278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:10.494543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.495489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.495928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.496353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.497818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:10.498308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.500674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.530345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 401/819] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:10.535194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.605682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 419/820] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:10.634853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.789211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4717de-8d31-4439-83ae-125e991c3732 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.820905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0e5b7-6409-4d58-8238-6a28f7f94b28 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.830829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.872429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.873272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.873548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.873816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.886479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.886479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:10.909897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:10.911180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:10.912344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:10.914641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.915720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.916162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.916575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] PATH_INFO: `/v3/services/network` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.917392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.919028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.923938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 402/821] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:10.927694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:10.937049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:10.938311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:10.939426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:10.942694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.955013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 420/822] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/services/network => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:10.975566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.156820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e56c-80d3-431f-9cc5-5c590b82d094 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.210287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588ead-6a5f-4ddc-af91-f37345c44ada None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.233673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.235524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.236146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.238075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.239173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:11.239910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.242593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.272362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 403/823] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:11.276705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.298460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.299487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.299960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.300392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.301263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:11.301713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.303305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.316249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 421/824] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:11.325004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.383228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.411225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.433726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df97f41-4620-48cb-8ea4-6be6b767cf2f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.474636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.475298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.475642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.475916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.556522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.558128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.584242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc054121-a343-4dcb-a9d2-b9532cfb77b2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.650753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.682374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 404/825] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:11.689333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.702431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.703129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.703422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.703700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.704684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:11.704993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.706403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.733486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 422/826] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:11.748093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.818134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee7de6-29aa-49f3-959f-bfe28757ddc5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.860440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.861642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.862128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.862633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.875898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.877765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.963281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.979590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeded84-709c-48fd-bdfb-e94a7c0feb50 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.042194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.042951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.043240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.043610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.044495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.046071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e969e-05a9-4671-9694-e6686c523392 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1161039a-2f68-40ae-8355-5ae7e56ab794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:12.049020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:12.050034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:12.051788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:12.051788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:12.051788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:12.051788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:12.053090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 405/827] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:12.058964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44377-6b77-4c95-865b-a9fcf2b777da None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.058964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44377-6b77-4c95-865b-a9fcf2b777da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.058964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44377-6b77-4c95-865b-a9fcf2b777da None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.058964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 406/828] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:12.067752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c44377-6b77-4c95-865b-a9fcf2b777da None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.083399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 423/829] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:12.085233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a52a4-7daf-4cc1-946c-d2ac988ecb7a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.085763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a52a4-7daf-4cc1-946c-d2ac988ecb7a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.086336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a52a4-7daf-4cc1-946c-d2ac988ecb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.114989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.118317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a77a52a4-7daf-4cc1-946c-d2ac988ecb7a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:12.119868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a77a52a4-7daf-4cc1-946c-d2ac988ecb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:12.225337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 424/830] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:12.226860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd05d-5ab2-4dc0-aa6e-749eb36a2296 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.227165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd05d-5ab2-4dc0-aa6e-749eb36a2296 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.227430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd05d-5ab2-4dc0-aa6e-749eb36a2296 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.229315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 425/831] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:12.235251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.235642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.235808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.255666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:12.257190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:12.268072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c44377-6b77-4c95-865b-a9fcf2b777da None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.331018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.332417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.333005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.333290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.334071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:12.334449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.336996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.350609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 426/832] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:12.351806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.352608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 407/833] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:12.358882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.366650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.460024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.511857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0dbc0-99c0-4fba-a603-8a73e3bd28e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.533188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c430436-2227-4b43-b378-00bf8045e9d7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.556445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.557278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.557562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.557839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.567363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.568919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:12.585952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:12.587018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:12.587912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:12.592452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 408/834] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:12.593508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.708629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.709952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.710479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.710975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.713277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fc622-c37b-4416-884f-98f33ef5a7ec None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.731183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:12.745691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:12.747955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:12.749392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:12.750333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:12.751204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 427/835] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:12.752141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.757145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.758112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.758503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.758861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.759987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.761673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a18284c5-ebca-40e6-b2f0-b4930c8ff70a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:12.774778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:12.784322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:12.786704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 409/836] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:12.789394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187cab-2fe8-4d15-ab7d-8b5239ec52da None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.789394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187cab-2fe8-4d15-ab7d-8b5239ec52da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.789394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d187cab-2fe8-4d15-ab7d-8b5239ec52da None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.789394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 410/837] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:12.789394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d187cab-2fe8-4d15-ab7d-8b5239ec52da None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.848878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.894762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a2c7-c50b-4101-9352-0eff1e4db72a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.937500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d187cab-2fe8-4d15-ab7d-8b5239ec52da None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.942847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.943904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.944362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.944811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.946186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:12.946734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.949283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.976207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 428/838] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:12.976207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.997315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.998363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.998855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.999308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.000412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.000916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.003398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15344-af4a-4b7e-b6a0-5e4f36b95eb8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.023453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 411/839] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.023453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.023453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.023453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.049184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:13.051109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:13.128242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.130656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf5f0-e559-4a90-a0a3-349e0720a4fe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.150537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 412/840] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:13.151828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.176203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.188767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.189590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.190005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.190398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.191272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.191697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.193372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.200738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 429/841] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.202459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.296091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc76c7-b359-4b99-b64e-7222b4671ad3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.335028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a0738-7b77-4c44-8aa5-b58bbea1a3da None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.339402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.340210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.340613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.341502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.342647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.343060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.345052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.354085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 413/842] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.355590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.390972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.392110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.392585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.393080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.413268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.415480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:13.439241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:13.441039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:13.442883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:13.442883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:13.442883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:13.442883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:13.442883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 430/843] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:13.443414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.465443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.522905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.559075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fc78-35ff-4fad-a207-9aa6579ba44c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.624745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.626079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.626543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.627336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.642523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b855a-db63-4f71-85e6-5be8ea246b5e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.648278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:13.664943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:13.666104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:13.667025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:13.668233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 414/844] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:13.670101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.691999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.693091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.693649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.694457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.695534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.696130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.697856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.706063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 431/845] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.708694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.909592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c499c-9f02-4d6f-972e-f276a863b4c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.927512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.952859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7b0d-9cf2-4276-9e8d-6259b3fc357c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.977846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.979133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.979664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.979784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.980298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.982300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.982960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.985724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.012143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 415/846] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.024191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.037859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.038496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.039196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.040815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.041478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.044747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.060934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 432/847] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.068874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.251935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086896b4-efb1-4e6b-8cf5-9d8e64139be0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.279805 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608329d0-7d1a-4341-adec-f6c18fa2beda None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.327152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.328169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.328699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.329082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.345340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.347049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.349720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.352013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.352702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.353403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.368817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:14.369539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:14.371540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:14.374145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:14.374145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:14.374145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:14.374145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:14.374145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 433/848] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:14.377372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.382905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:14.413246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:14.415414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:14.417620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:14.422699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 416/849] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:14.426470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.438744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.489814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.607975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe00b25-4105-4528-829f-8d5f6a1f3b65 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.665532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a71271-d53b-4556-afa3-bd82b736a962 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.690922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.692462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.693223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.693941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.695786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.696642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.699888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.726678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 434/850] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:14.736446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.744883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.746743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.747632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.748402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.750491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.751388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.754656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.793242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 417/851] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.794544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.903970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.918201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71138fcd-36a8-4378-8ddd-13965ad06529 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.937479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d28c-ff64-44e4-baac-395a8b5bebe6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.960211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.963302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.964247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.964995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.965634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.007141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.008672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.009379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.010088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.027198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.031105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.057710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.059226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:15.062638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:15.064621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:15.067543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:15.076514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 435/852] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:15.076514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.185415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.202584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 418/853] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:15.208607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.298303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c78f8-2c06-48a4-bbcb-4930bf1a60de None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.348243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721440b2-f849-4996-9ef6-e4e5c1709bda None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.370867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.372692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.373329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.373828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.375749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:15.376312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.379445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.392797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.393786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.395205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.395205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.400240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.414165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.416900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.435006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 436/854] 10.222.0.77 () {56 vars in 1135 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:15.439301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:15.441221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:15.443319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:15.445447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:15.445447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:15.445447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:15.445447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:15.447267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 419/855] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:15.451563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.513825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.594444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fa0d2-e0a4-48ab-a5b5-2c902ca92485 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.644522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.645553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.646045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.646574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.647527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:15.648039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.650470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.666368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5610acf-3c55-42e4-84ed-7dada9da2568 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.670674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 420/856] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:15.674363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.741088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.742150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.743072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.743324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.819163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce45142-d609-4b18-bd39-166206c221df None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.822397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.859305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.864514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.866787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.892722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.894065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.894589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.895084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.910398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.914250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:15.945185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:15.946313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:15.947388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:15.954553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 421/857] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:15.957922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.043058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 437/858] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:16.047144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.087886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5ad23-6c19-4e6d-b587-50e57eb7041f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.134855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.136532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.137294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.138144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.139243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.139863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.141967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.166551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 422/859] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:16.173444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.226881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.232377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.245497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd5e78-f052-4f99-a7eb-49ba890dea0d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.315892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.316621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.316928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.317174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.330389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.332110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.343707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48d958-1c42-42cb-81be-7048c980e628 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-43d3a384-0fe4-4653-9d08-22855914d9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:16.352475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:16.359633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 438/860] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473effd-9b15-40a3-b493-529c35f4db04 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473effd-9b15-40a3-b493-529c35f4db04 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473effd-9b15-40a3-b493-529c35f4db04 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 439/861] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:16.367666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.367666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.367666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.392729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:16.394907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:16.425203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.426285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.426820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.427313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.429073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.429474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.432429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.452235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 423/862] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/role_assignments?user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:16.452235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.510919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 440/863] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:16.527318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.611276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.635945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d95f-7dd2-449e-a21a-926faa32c4ae None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.692899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.706576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.708103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.708772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.709415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.710845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.711362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.714586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80580c-da10-4bea-bb1b-8ac6a429cc3e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.729285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa243-c3ef-4b90-b485-552b50d776a8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.735837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 424/864] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:16.739969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9f44-b172-44db-8eaa-be01963dc621 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.740836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9f44-b172-44db-8eaa-be01963dc621 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.741723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9f44-b172-44db-8eaa-be01963dc621 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.744510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 425/865] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:16.745964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9f44-b172-44db-8eaa-be01963dc621 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.798795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.799893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.800292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.800818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.801684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.803889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904dde2-15f9-4f0d-849c-d7917f89cc75 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.823487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 441/866] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:16 2022] POST /identity/v3/services => generated 251 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:16.825659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.826010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.826320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.868373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:16.871239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:16.989939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9f44-b172-44db-8eaa-be01963dc621 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.030363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 442/867] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:17.075270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.081748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.082202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.082619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.083062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.100385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.104418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:17.133131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:17.134604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:17.135687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 426/868] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:17.145001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.149998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.225377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.304411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3445b55-a0c1-48d3-95eb-768664911fe0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.380680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ebc-ea11-4dc3-983a-d56e4166ef7f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.387757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.392124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.392998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.393692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.395257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.398533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:17.425089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:17.426965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:17.428589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:17.434141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 443/869] 10.222.0.77 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:17.450477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.458780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.460046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.460625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.461191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.462845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:17.463517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.467228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.499540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 427/870] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:17.505472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.642066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.711935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cf415-953d-4edd-8884-3c086f00a509 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.744641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed644-8e12-415b-84e9-aa8a5211fb35 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.786344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.787961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.788471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.788963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.790281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:17.790886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.794274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.796372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.812011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 444/871] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:17.817783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.838918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.841158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.841911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.843295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.845789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=71aecafdbfad40d4979f6f6358a58109) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:17.846724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.851136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fc663-6881-4603-b4d7-0bfc3054b7f4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.866800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 428/872] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/role_assignments?user.id=71aecafdbfad40d4979f6f6358a58109&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:18.029810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3d9f2-cbcb-45e0-906d-58a3a9c2c0c1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:18.098136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:18.099908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.100765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.101553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.103299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:18.104198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:18.107757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a96bf-114a-4679-935f-605536a88a84 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:18.117724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 445/873] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:19.848883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bd101-1ae4-4f29-a72f-341ed7a0987e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.849412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bd101-1ae4-4f29-a72f-341ed7a0987e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.849768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bd101-1ae4-4f29-a72f-341ed7a0987e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.852728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 429/874] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:19.861931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.863470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.864210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.894921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:19.897367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:20.019502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.042289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 446/875] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:20.087688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bd101-1ae4-4f29-a72f-341ed7a0987e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.305070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bd101-1ae4-4f29-a72f-341ed7a0987e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.359990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.377732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.379171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.379670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.380173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.401504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0f8a28a9-445e-4463-9e6f-807ff08993ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:20.419604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:20.421944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:20.423899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 430/876] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:20.436107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.516197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff41b53-b69b-4ee6-8a6f-b54b63d9a4c6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.517179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff41b53-b69b-4ee6-8a6f-b54b63d9a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.517960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff41b53-b69b-4ee6-8a6f-b54b63d9a4c6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.521407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 431/877] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:20.531125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c90802-db36-4b64-9956-2950911223cd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.532199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c90802-db36-4b64-9956-2950911223cd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.532973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c90802-db36-4b64-9956-2950911223cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.563984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c4c90802-db36-4b64-9956-2950911223cd None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:20.566572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c4c90802-db36-4b64-9956-2950911223cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:20.666622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b4aef-9430-4dbc-80f2-b6923aaecbc0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.730066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 432/878] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:20.744601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.746261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.747175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.747939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.749673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:20.750548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.753526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.772186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c90802-db36-4b64-9956-2950911223cd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.774566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 447/879] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:20.785711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.895371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.953970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed8eb-d392-43de-8de7-a5be2e0ad72d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.994220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c90802-db36-4b64-9956-2950911223cd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.997817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.999199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.999698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.000588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.016548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.018076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.029552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a58cba6f-58dd-477b-8351-f2da780aa97b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:21.040497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:21.041934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:21.044706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:21.044706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:21.044706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:21.044706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:21.044706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 448/880] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:21.045551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbe1dc-4bbc-423a-9928-b61d9c0b02d8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.045551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbe1dc-4bbc-423a-9928-b61d9c0b02d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.045906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbe1dc-4bbc-423a-9928-b61d9c0b02d8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.054220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 449/881] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:21.054220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbe1dc-4bbc-423a-9928-b61d9c0b02d8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.075719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.077341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.077807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.078258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.079535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.081564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6b952759-0692-4477-a07c-a62a94f60ba4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:21.102202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:21.107272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 433/882] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:21.108885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.108885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.108885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.138256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:21.139644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:21.239882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 434/883] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:21.240884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.280007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbe1dc-4bbc-423a-9928-b61d9c0b02d8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.290703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.333395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.334177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.334686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.335098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.336231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.336718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.338385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec62fb-84bb-4fe6-a1c9-c936b169cc16 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.353339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 450/884] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:21.355088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec09af-4d25-4884-bfbf-6fdc6259b219 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.355550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec09af-4d25-4884-bfbf-6fdc6259b219 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.356028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec09af-4d25-4884-bfbf-6fdc6259b219 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.361403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 451/885] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:21.361403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec09af-4d25-4884-bfbf-6fdc6259b219 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.394103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca245d-8d36-4b81-8f9c-4361b6b27cfb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.413861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.446890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.447694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.448037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.448371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.449161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.449525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.451215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.457335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 435/886] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:21.458670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.511560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec09af-4d25-4884-bfbf-6fdc6259b219 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.561495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.562396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.562773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.563090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.563920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.565469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3295a-af95-44d0-8d34-9f6c628c926c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.597210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 452/887] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:21 2022] POST /identity/v3/services => generated 269 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:21.597210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.597210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.597210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.614353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53214f98-41a0-4166-92fd-9f957b8318ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.628336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:21.630497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:21.668481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.689567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.690987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.691535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.691944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.709159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.712193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:21.733700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:21.735554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 436/888] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:21.737297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.767753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 453/889] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:21.768005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.802288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.924199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bcd80-ee4e-4a0d-a18e-a475f32da7d8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.933726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52da5fd-4f2b-47db-9259-80d7c0a4ad15 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.974791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.975767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.976242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.976723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.977989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.978477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.980381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.005129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 454/890] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.006121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.010911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.012308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.013283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.013765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.014880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:22.015366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.016960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.030208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 437/891] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.030208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.040908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.172971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.186019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110870e-da03-4458-8095-e780bdeceeeb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.216889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9f4d9-500d-46f7-8788-610a74bcfcd5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.249839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.250933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.251407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.251877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.288766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.290214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.290811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.291275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.292522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.295153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:22.319215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:22.320648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:22.321510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:22.330885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 455/892] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:22.332187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.352299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.354168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.491982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.503531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.521821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 438/893] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:22.523486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.534020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ee0d9-1e72-4c21-8e7d-c60a6791bb7f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.583591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.584857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.585574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.586014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.587409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] RBAC: Adding query filter params (interface=public, service_id=8436abfdcf89486abf0be3b7f169ca99, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:22.587880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.589491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.601989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 456/894] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/endpoints?service_id=8436abfdcf89486abf0be3b7f169ca99&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.603492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.728975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ea9d-e64a-47bb-82d7-0e00fbea1f53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.777616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f569c-82c1-4f8f-aa47-0f8a5aa098a0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.783353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.783881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.784159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.784429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.785165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:22.785469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.786950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.792997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 439/895] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.793614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.852476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.854241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.855083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.855587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.879907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.880840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.882703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.888646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:22.923149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:22.925549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 457/896] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:22.928699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.946565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ce086-0e8b-424c-911f-7165df9d83b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.008752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.009903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.010382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.010908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.012441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.015079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.026594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 440/897] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services => generated 1268 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:23.028148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.131545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553e868-ad88-4ea4-9684-aeea66e22dce None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.212093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.213163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.213661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.214157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.215625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:23.215944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.217965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.235723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 458/898] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:23.266802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f1e9-3c7e-46b0-8b26-c958cb90d583 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.312735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:23.337865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.339671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.340605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.341362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.343151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:23.344077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.347181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.372443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 441/899] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:23.381361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.510950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:23.597592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc80b33-040f-412b-b683-f8ce4d98aa99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.665498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.666940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.667443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.667936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.682073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.684968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a961502d-b62e-4e59-be70-4a22da9a8e6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:23.713913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:23.716257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:23.717989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 459/900] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:23.726760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.823605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba252e-263f-4127-a094-c8f105caf2f5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.824041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba252e-263f-4127-a094-c8f105caf2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.824394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba252e-263f-4127-a094-c8f105caf2f5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.828040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 460/901] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:23.833832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.834231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.834524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.856077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:23.857575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:23.912229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46923e-2580-4549-a1b9-f1e65abaee06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.958630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 461/902] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:23.983712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.988287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.988960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.989247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.989530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.990600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:23.990920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.992830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.007934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 442/903] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.014763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.086191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.096958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.163056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70110e5-03d0-4f71-b251-46b3ca91c3d7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.203443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.204394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.204678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.204945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.205706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.207597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:24.220601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:24.222546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:24.224242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 462/904] 10.222.0.77 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:24.235511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.244669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba540635-2bcf-45f5-9fc3-6f42785e305c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.326123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.327329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.327815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.328455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.330441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] RBAC: Adding query filter params (role.id=1a3770e02a3e474e938ba08f11c36fbe, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.331187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.334166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.346846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 443/905] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/role_assignments?user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=1a3770e02a3e474e938ba08f11c36fbe => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:24.377616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13d29e-6633-4403-843d-53bdb48562e7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.407837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.453039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.454515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.455019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.455521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.456879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.457389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.460057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.470603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 463/906] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.484099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.695734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255887-b08a-4cdb-bc7b-c30dc49b3182 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.712185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.751212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.752054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.752435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.752714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.753449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.753754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.755343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.761808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 444/907] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.772133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.920568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ca94-4769-4f55-9413-77ef85d4aaab None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.961369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.962153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.962320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.962636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.963454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] RBAC: Adding query filter params (interface=public, service_id=13c99c28a3e244ab890d06eccdcb40e6, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.963769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.965200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242f18e-57f4-40cf-8cfd-67913180b497 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.971597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 464/908] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/endpoints?service_id=13c99c28a3e244ab890d06eccdcb40e6&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.979750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:25.184850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e67b51-a729-46b5-bbbf-5e49abfc09ff None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.256568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.257728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.258287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.258819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.260228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.263210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714be73f-9b81-49e6-b508-a5ac56ab12c6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.275873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 445/909] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services => generated 1268 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:25.779144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e756f8-dd62-4de6-852e-d93c9ed7f1f2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.779432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e756f8-dd62-4de6-852e-d93c9ed7f1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.779719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e756f8-dd62-4de6-852e-d93c9ed7f1f2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.781448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 465/910] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:25.788122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.788864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.789481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.814233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:25.815476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:25.860393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.941148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 446/911] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:25.983268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e756f8-dd62-4de6-852e-d93c9ed7f1f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.066467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:26.138030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e756f8-dd62-4de6-852e-d93c9ed7f1f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.178226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.179264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.179546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.179822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.180493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.181907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-98d72919-a0e6-4552-86e8-ed1c9278cf3d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:26.194842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:26.195867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:26.197026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 466/912] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:26.206752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.310755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141634bf-2eba-4b0c-b17c-04327d525569 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.311208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141634bf-2eba-4b0c-b17c-04327d525569 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.311582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141634bf-2eba-4b0c-b17c-04327d525569 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.313377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 467/913] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:26.319245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.319688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.320157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.339621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:26.341766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:26.393836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27600ac9-e3bf-47a2-9ec9-3eca8b789183 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.439331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.440265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.440658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.441025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.441425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 468/914] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:26.443914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:26.444479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.447256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.457829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 447/915] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:26.466708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.466913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.509204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:26.532569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:26.606262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263728-86b8-4ac1-9a97-f212c9141b64 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.663304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.663996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.664279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.664556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.665718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.667849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.681573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003781a-0ac5-43f2-bf66-d901b1243dee None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.686017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 448/916] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:26 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:26.772563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.773521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.773968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.774525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.775831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:26.776329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.779381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f85ce-4a95-461a-8a9e-e582a9457814 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.788950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 469/917] 10.222.0.77 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:26.797088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.979235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.007604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c52aa-2a07-4170-9c92-e504b583213f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.078486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.079766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.080261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.080744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.082226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.084952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fd1e4-a66e-412d-907c-c1e7ee6530f4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.123329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 449/918] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:27.514939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583008e-634d-468f-90d1-e1e0a3fd86be None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.515670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583008e-634d-468f-90d1-e1e0a3fd86be None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.515670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583008e-634d-468f-90d1-e1e0a3fd86be None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.517384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 470/919] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:27.523557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2e9a-36c2-48a6-b108-f80dd5dcac23 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.524297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2e9a-36c2-48a6-b108-f80dd5dcac23 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.524297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2e9a-36c2-48a6-b108-f80dd5dcac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.544074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a83a2e9a-36c2-48a6-b108-f80dd5dcac23 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:27.545313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a83a2e9a-36c2-48a6-b108-f80dd5dcac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:27.667547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264b54-785f-4777-ab49-f53719bda010 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.668042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264b54-785f-4777-ab49-f53719bda010 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.668572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4264b54-785f-4777-ab49-f53719bda010 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.671215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 450/920] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:27.671376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 471/921] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:27.681050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.681545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.682007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.695844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4264b54-785f-4777-ab49-f53719bda010 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.711263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:27.713341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:27.837683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.856090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 451/922] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:27.878593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.896227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.896993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4264b54-785f-4777-ab49-f53719bda010 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.970451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.971776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.972254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.972734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.973975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.977194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:28.005162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:28.007294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:28.008463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 472/923] 10.222.0.77 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:28.020235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.113329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2eaa9-a996-4152-aeb7-99af60991ad0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.172626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.173955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.174555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.175021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.192223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:28.207353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:28.208359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:28.209205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:28.210020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 452/924] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:28.219490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.234754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b60e7-073c-451c-bb39-aaaccf218c8a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.285170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.286851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.287459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.288213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.289167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:28.289703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.292377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.305445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 473/925] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:28.308427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.345800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.362674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.462840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90b70b-b977-425e-9243-adbfb67fa056 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.463222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247fdbf-0c06-4deb-a925-ecd5a9dd3c27 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.508295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.508976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.509260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.509534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.510296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:28.510635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.512090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.522080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 453/926] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:28.530895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.544569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.545740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.546240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.546777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.548350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:28.548908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.551799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.563188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 474/927] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:28.573592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.753209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dbc3-e11f-4321-b43e-67c58445e2c0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.761518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.785926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.798581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae6e0-f15f-42ac-970f-5fc02e2763cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.800594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.801413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.801694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.801968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.814646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.815844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:28.835800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:28.837543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:28.839395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:28.839395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:28.839395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:28.839395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:28.842648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 454/928] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:28.844503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.878555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.880531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.881372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.882193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.884283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.888197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.929316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 475/929] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:29.008070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f8545-6bb9-4928-9005-eac57a34bbff None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:29.067931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:29.068783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.069101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.069474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.070250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:29.070594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:29.072120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:29.094046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 455/930] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:29.096553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:29.288605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761f32-1366-484e-9c81-3ed1622ce80b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:29.358139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:29.359587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.360093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.360600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.376525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:29.379429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:29.410269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:29.411321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:29.412274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:29.413301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 476/931] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:29.420350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:29.456336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:29.540182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc12a8d-9c06-447a-91e6-2be189ab82ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:29.583295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:29.583989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.584279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.584528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.585396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:29.585702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:29.590816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a379-8638-4844-84b5-8ff85db8aa59 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:29.614021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 456/932] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:29.615554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:29.618637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:29.763321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5df2-e423-4a15-b78e-d3eed0abaf4c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:29.804972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:29.805689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.805991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.806241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.819339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:29.820857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:29.841771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3da1c-e677-4b63-bc58-c15b8a82b87a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:29.842263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:29.843976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:29.845787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:29.845787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:29.845787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:29.845787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:29.846233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3da1c-e677-4b63-bc58-c15b8a82b87a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.846233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3da1c-e677-4b63-bc58-c15b8a82b87a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.848565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 477/933] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:29.849808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:29.850883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 457/934] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:29.855072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.855568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.856609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.885167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:29.887373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:29.936317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.027865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 458/935] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:30.051546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643bab-fbfd-44d5-9f3b-689a5cce6fbe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.073449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.113223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.134890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.136311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.136832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.137391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.139544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.140133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.143228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.173349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 478/936] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:30.179656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.253073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f710d5-db6e-4e0e-8c3d-78fb4f832d35 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.297199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.297847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.298131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.298398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.299135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.300614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:30.314523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:30.316528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:30.318278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 459/937] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:30.328312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.428605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521350e2-6d60-4f65-89fe-344d2e626b5d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.455171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.463472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.473920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65964f54-f25e-4518-b790-bc9a1c2eac35 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.508061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.509459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.509972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.510480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.523203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.524593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.525089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.525462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.525807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.526844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.527365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.529097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.529790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.537067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 460/938] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:30.545131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:30.561095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:30.562104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:30.563022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:30.570618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 479/939] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:30.573824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.715950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45498ecc-5d73-4c89-bad9-18263dba4e2f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.780531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.781686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.782178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.784010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.784274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b3e5-9f29-4cee-8bf0-6340197e3baa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.784731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.784731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.787334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a4287-24e7-42b7-9282-cde43f9bf9ea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.802654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 461/940] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:30.802654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a3b0-7b25-495a-871c-0064ef99950d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.802654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a3b0-7b25-495a-871c-0064ef99950d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.802654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a3b0-7b25-495a-871c-0064ef99950d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.804208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 462/941] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:30.805732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a3b0-7b25-495a-871c-0064ef99950d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.829981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.846552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.847231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.847507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.847782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.848616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.849674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.851189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c262-6206-476d-9b50-76dc980fec0a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.876522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 480/942] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:30.878901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.879433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.879937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.909837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:30.911862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:30.967049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.015155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a3b0-7b25-495a-871c-0064ef99950d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.031577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 481/943] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:31.036067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.070708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.071809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.072273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.072800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.074143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] RBAC: Adding query filter params (interface=public, service_id=21a36811fba647be871ff4f8a195dddd, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:31.075235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.077196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.088884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 463/944] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/endpoints?service_id=21a36811fba647be871ff4f8a195dddd&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:31.088884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.217335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.240702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92d54f-1736-4266-917e-fafc3ec43c9c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.256723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683702-d206-4ef6-9d95-76929b16fcbd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.306207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.307941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.308415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.308922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/e98cf2fa9bee4dcfa9f7410d19b67f7e/roles/1a3770e02a3e474e938ba08f11c36fbe` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.312832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=e98cf2fa9bee4dcfa9f7410d19b67f7e, role_id=1a3770e02a3e474e938ba08f11c36fbe)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.313065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.313940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.314385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.314872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.315442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.316094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.318646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.320999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING keystone.server.flask.common [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 16:25:31.354450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 482/945] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:31 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/e98cf2fa9bee4dcfa9f7410d19b67f7e/roles/1a3770e02a3e474e938ba08f11c36fbe => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:31.354450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.360473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:31.360473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:25:31.360473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [parameters: {'id': 'b211f39a94874487be4c48fe864fa209', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:25:31.360473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b211f39a94874487be4c48fe864fa209.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b211f39a94874487be4c48fe864fa209. Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:31.398066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 16:25:31.399683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() Jul 18 16:25:31.402961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:25:31.406396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:25:31.410368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:31.414240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': 'b211f39a94874487be4c48fe864fa209', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:31.418048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:31.424907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b211f39a94874487be4c48fe864fa209. Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 464/946] 10.222.0.77 () {60 vars in 1140 bytes} [Mon Jul 18 16:25:31 2022] POST /identity/v3/projects => generated 262 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:31.428601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.431167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.582257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac907c1-ecc2-4ea8-b433-646809a97a5c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.634510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e52e2-9e3e-4c0a-a7b4-007c59bde291 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.659353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.660229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.660659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.661057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.664114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.665860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636bac7-446f-42af-92df-4e8b72bd4393 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.678197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 483/947] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:31 2022] GET /identity/v3/services => generated 1497 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:31.708702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.709836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fc594-ddd9-4e83-9e8b-6283cbcae9e5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.711946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fc594-ddd9-4e83-9e8b-6283cbcae9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.712893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fc594-ddd9-4e83-9e8b-6283cbcae9e5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.717053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 484/948] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:31.721981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.722924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.723213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.723488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.723626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.724055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.724575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.737019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.738910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.750925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:31.753387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:31.755649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:31.757248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:31.758887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 465/949] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:31.772259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.925483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 485/950] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:31.957499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.999326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:32.007366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423a27e-abaa-4b96-9c0e-7993ae9253c4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:32.073581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:32.074787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.075271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.075838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.077272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:32.077918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:32.080963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:32.130109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 466/951] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:32.169228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa611a-7bea-4b8c-9897-21684ee76d47 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:32.228554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:32.229278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.229556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.229827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.231102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:32.232638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:32.246837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:32.248937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:32.251621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 486/952] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:32.265047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:32.383106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:32.476599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73292940-f0be-4024-a2d2-cecdd6369a98 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:32.532831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:32.534729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.535449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.536212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.537921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:32.538826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:32.541785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8c6c-1db9-4531-bbf8-c384fed0ab7b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:32.553155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 467/953] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:32.563435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:32.686647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:32.727524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb75a-209c-495b-8608-4df29648db55 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:32.798717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:32.801826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.802741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.803506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.805239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:32.806178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:32.809183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f17e2-8089-4d14-858e-3249af1cc463 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:32.819270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 487/954] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:34.375446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e817a5d-449a-4ca2-95bd-f0fed96dfa27 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.376306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e817a5d-449a-4ca2-95bd-f0fed96dfa27 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.376306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e817a5d-449a-4ca2-95bd-f0fed96dfa27 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.377832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 468/955] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:34.383976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905112-ecc7-49db-8557-384a4a641062 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.384751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905112-ecc7-49db-8557-384a4a641062 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.384751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49905112-ecc7-49db-8557-384a4a641062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.404038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-49905112-ecc7-49db-8557-384a4a641062 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:34.405326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49905112-ecc7-49db-8557-384a4a641062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:34.496192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 488/956] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:34.522343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e817a5d-449a-4ca2-95bd-f0fed96dfa27 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.556356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e660a-17c9-43ad-b49f-5566f8d3ceda None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.556650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e660a-17c9-43ad-b49f-5566f8d3ceda None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.556987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e660a-17c9-43ad-b49f-5566f8d3ceda None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.559257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 489/957] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:34.572161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.572470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.572743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.593472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:34.594770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:34.720138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e817a5d-449a-4ca2-95bd-f0fed96dfa27 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.721354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 490/958] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:34.752139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.760457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.771727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.806859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.809311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.810078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.810822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.812380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.815578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:34.843071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:34.845310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:34.847128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:34.851714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 469/959] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:34.858476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.969412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa62dd6-68ec-4532-8888-7367006a71c8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.981335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8918b8-325a-40bb-8f7d-2eb351894cce None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.024109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.025067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.025595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.026062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.027117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.027762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.029749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.041871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 470/960] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:35.044236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.044958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.045942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.046606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.047184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.069990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:35.087974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:35.089983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:35.091719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 491/961] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:35.105017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.137629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.218103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.237406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1083ee-5626-462a-b2d3-9c7606438327 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.282585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.283850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.284703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.285244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.286367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.286939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.288809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.302196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 471/962] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:35.303617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.304840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384d7e-63ca-4f95-a241-ee7db847be22 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.379691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.380837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.381352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.381847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.382888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.383416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.385020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.401249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 492/963] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:35.403205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.471986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b98f7e-dbb6-494c-8ed6-4768d3ebc211 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.483390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.522133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.523220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.523724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.524195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.525353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.527129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47784ae1-8580-4411-b391-a7b0417123bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.572292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 472/964] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:35.606039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d5ae-649b-4253-98eb-95d345d7cd58 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.606791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d5ae-649b-4253-98eb-95d345d7cd58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.607272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d5ae-649b-4253-98eb-95d345d7cd58 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.609297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 473/965] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:35.616848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e499d-957b-491d-9147-8fe6868c1a17 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.619979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.621100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.622100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.662910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.666144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:35.667937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:35.686894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.687950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.688239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.688511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.704562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.706359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:35.727658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:35.729430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:35.732814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:35.732814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:35.732814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:35.732814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:35.734033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 493/966] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:35.736605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.827262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 474/967] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:35.869304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.934698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d376-b242-48af-b5a9-738de2a06703 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.004741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.005913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.006390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.006982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.008416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:36.008972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.011592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.035508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 494/968] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:36.044800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.045215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.066237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e9f19-b99d-45ce-9639-9d411219cfc9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.094178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.136822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.137819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.138395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.139041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.140633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.143682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.166906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:36.166906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:25:36.166906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [parameters: {'user_id': 'fc9f703e40b94036ac4ff0674f0fb770', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:25:36.166906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:36.185184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:36.186230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:25:36.187285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:25:36.188195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:36.189082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:36.189943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fc9f703e40b94036ac4ff0674f0fb770', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:36.191125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 16:25:36.192218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.193316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 475/969] 10.222.0.77 () {60 vars in 1131 bytes} [Mon Jul 18 16:25:35 2022] POST /identity/v3/users => generated 160 bytes in 321 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:36.194396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a259b69-4b4f-4285-a6e7-c97a04fe3ce8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.199529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.239049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.239877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.240176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.240520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.248961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.250446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b068b0c7-9657-43d5-b21e-4aea26994ed4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:36.267012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:36.268084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:36.269084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 495/970] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:36.269708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bfd2-79e1-4ae8-a351-62f0bd6610f3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.270314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bfd2-79e1-4ae8-a351-62f0bd6610f3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.270314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bfd2-79e1-4ae8-a351-62f0bd6610f3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.271617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 496/971] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:36.277054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.277937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.278216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.298992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:36.300225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:36.368959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26cfd6-aadf-440d-a2ff-a86210a6ec67 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.419806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.420732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.421111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.421390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.431056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.432750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.438327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 497/972] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:36.440034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:36.468479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:36.469695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:36.471307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 476/973] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:36.472951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.493766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.499534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.604128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768967-2576-4e9a-87a9-f0f8770ecf1c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.654030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcae18-99d2-4596-b918-df3b79316002 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.678089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.679312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.679814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.680313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.681858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:36.682168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.685597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.713288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.714149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.714597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.715072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.715963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.717512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.726150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 498/974] 10.222.0.77 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:36.727722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.745909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 477/975] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:36 2022] POST /identity/v3/services => generated 242 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:36.748399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.879251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.922209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b6fa2-d5af-4f92-85b4-7706cbfa5ace None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.933002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.964122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.965362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.965705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.966070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.972152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b51bd-507e-42ba-9b9b-73e724df94e8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.024804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.025839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.026299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.026860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.028395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:37.028867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.030543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.070697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 499/976] 10.222.0.77 () {56 vars in 1101 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:37.090426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.092675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.235208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:37.298737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 478/977] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:37.304378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:37.433772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0767201-e70d-436c-8d5a-4c6462353a69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.490804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.492095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.492600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.493088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.514804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.516874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.520279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:37.551750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:37.552875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:37.553964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:37.553964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:37.553964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:37.553964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:37.554483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 500/978] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:37.560988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:37.781183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f508d-688f-4e3f-b52f-8b2d54eaff55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.847834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.848979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.849438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.849906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.850897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:37.851210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.853062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.882454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 479/979] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:37.886076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.074097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315da85-1664-4f67-bd0c-90fe5c9e0355 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.117584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.118456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.118762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.119039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.128264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.129788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-093f8b12-8e5d-496d-a7b5-d0b5f5e5e8b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:38.155888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:38.157922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:38.160064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 501/980] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:38.165394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.204448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.221195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ba477-239b-4aef-ae1e-5e3af23057e2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.221484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ba477-239b-4aef-ae1e-5e3af23057e2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.221767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ba477-239b-4aef-ae1e-5e3af23057e2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.223456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 502/981] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:38.229429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.229725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.230062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.252299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:38.254392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:38.311008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f918-5f33-4fe9-8f77-fb2b6c1192e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.328287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.349893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 503/982] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:38.364550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.365378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.365764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.366139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.367118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:38.367531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.369132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.384243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 480/983] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:38.388747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.390422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.554187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.576331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb3efd-d857-4603-884d-92d0afe95826 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.582687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d156b8-5c66-4e6c-af31-0fbaf932560a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.641953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.643515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.643978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.644365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.645845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] RBAC: Adding query filter params (role.id=1a3770e02a3e474e938ba08f11c36fbe, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:38.646552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.649489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.656417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.657783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.658268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.658790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.660013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.662168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 481/984] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/role_assignments?user.id=e98cf2fa9bee4dcfa9f7410d19b67f7e&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=1a3770e02a3e474e938ba08f11c36fbe => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:38.662586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:38.692242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:38.693357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:38.694258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:38.695270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 504/985] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:38.704321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.911334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db3ef-58c5-4d39-802a-4724f50862d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.961124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.961816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.962094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.962364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.963126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:38.963431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.965412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1cc05-de30-4cfc-acff-48db368ced97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.971978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 482/986] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:38.978762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.071919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.175980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4527-87d8-4888-b851-46e893016ed1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.176306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4527-87d8-4888-b851-46e893016ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.176646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4527-87d8-4888-b851-46e893016ed1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.177648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031734df-512b-45e0-90fb-eda5a3da2b3b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.178390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 483/987] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:39.187886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.188392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.188894 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.219364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:39.222163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:39.242488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.243357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.243778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.244260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.245344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.245790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.248225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.251003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.256736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 505/988] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:39.263888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.371914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 484/989] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:39.409899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.432875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4d41-0f4c-4683-814f-298fa4dc31ea None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.456815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.493225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.493886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.494346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.494829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.496673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] RBAC: Adding query filter params (interface=admin, service_id=21a36811fba647be871ff4f8a195dddd, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.496673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.498597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.515648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 506/990] 10.222.0.77 () {56 vars in 1249 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/endpoints?service_id=21a36811fba647be871ff4f8a195dddd&interface=admin®ion_id=RegionOne => generated 193 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:39.518994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.616726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21095b6-e66b-466d-8cec-657df587db6c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.688190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.690346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.690928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.691412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.692707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.695450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.699644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0f5a7-b2ba-48c3-ae2d-bbb945361961 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:39.720424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:39.721602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:39.722557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:39.723945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 485/991] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:39.733023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.753286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.754305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.754931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.755712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.756646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.758695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.763178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.770016 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 507/992] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/services => generated 1727 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:25:39.954457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dae52-cd1b-4203-a79c-edd739c7b7e5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.023119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.024351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.024846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.025366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.026909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:40.027771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.030144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.038216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 486/993] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:40.044907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.132029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.255397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e127-c746-4c7e-b5bc-33471fc35fbc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.323872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.325020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.325485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.325948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.327253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:40.327749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.330619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.340551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 508/994] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:40.349817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.504632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.553127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3317a-0bc3-44e9-bec3-9badb9830f5a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.627537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.629897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.630771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.631572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.633353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] RBAC: Adding query filter params (interface=public, service_id=2e8923659b744a83838811049ca342bc, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:40.634240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.637268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d58a-2dfe-4f95-9dd7-9c95f176db7d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.657365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 487/995] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/endpoints?service_id=2e8923659b744a83838811049ca342bc&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:40.659272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.870548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad865ce1-aed0-4104-97b6-5c527f470a77 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.875471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.942374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e601b4-a723-421e-8018-0c6e4bc7f8f3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.942701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e601b4-a723-421e-8018-0c6e4bc7f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.943023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e601b4-a723-421e-8018-0c6e4bc7f8f3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.944589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 488/996] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:40.946057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.947265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.947773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.948645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.949711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.950066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.950357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.950816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.953325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.969993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:40.978976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 509/997] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/services => generated 1727 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:40.979279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:41.066127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 489/998] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:41.090444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:41.259182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2cc98-bd1e-47f1-a04c-267cf9e33446 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:41.303646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:41.304452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:41.304741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:41.305008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:41.305670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:41.307158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:41.320463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:41.322328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:41.324503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 510/999] 10.222.0.77 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:41 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:41.335358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:41.424726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:41.465971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf51-c6b8-414f-9eb0-71fd4d8f39c6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:41.510126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:41.510968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:41.511438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:41.512400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:41.513289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] RBAC: Adding query filter params (name=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:41.513733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:41.515356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49045e38-295e-4909-b5ba-0f459c5e5612 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:41.527537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 490/1000] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:41.528831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:41.570236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:41.672117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3884-8843-4f85-8c7c-dbe937b998b5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:41.715416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:41.716152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:41.716434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:41.716675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:41.717398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] RBAC: Adding query filter params (type=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:41.717742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:41.719275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3405-3e80-43a9-94ca-73a40f89d098 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:41.725592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 511/1001] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:42.196442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4dd8-2073-4445-9eb1-ca900ca0f177 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.197265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4dd8-2073-4445-9eb1-ca900ca0f177 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.197265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4dd8-2073-4445-9eb1-ca900ca0f177 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.198709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 491/1002] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:42.204822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.205627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.205627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.225939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:42.227202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:42.313702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 512/1003] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:42.337699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b4dd8-2073-4445-9eb1-ca900ca0f177 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.470286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b4dd8-2073-4445-9eb1-ca900ca0f177 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.514469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.515569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.516007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.516461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.517292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.518920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.521532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:42.535806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:42.537782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:42.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:42.542294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 492/1004] 10.222.0.77 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:42.543944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.578464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.681372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2d4e6-5039-4c8c-abfd-3d17a3c4a074 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.724410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.724993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.725277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.725554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.726289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:42.726755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.728070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.738751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 513/1005] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:42.741137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.931886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91addc7-5ac6-4b30-8220-cdfcdf2e5ce2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.973740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.974526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.974731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.975032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.975702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:42.976005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.977403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb43844-bbb9-4377-af77-c089db7eff6b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.983796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 493/1006] 10.222.0.77 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:42.990141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.105868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.130501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae34472-0c4e-4a8f-908a-2d5788037aa0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.195909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.197043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.197544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.198038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.200184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.202798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f261c4b-30ba-42e3-9142-a9497990d45c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.230826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 514/1007] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:44.469260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019190-6adf-47ed-8164-99f849885c0a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.470074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019190-6adf-47ed-8164-99f849885c0a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.470216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019190-6adf-47ed-8164-99f849885c0a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.472945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 494/1008] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:44.482538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.483362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.483501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.497318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3298e-3039-456e-bde6-33ba60c0a7e6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.497823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3298e-3039-456e-bde6-33ba60c0a7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.498336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3298e-3039-456e-bde6-33ba60c0a7e6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.501265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 495/1009] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:44.509529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.510004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.510439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.513093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:44.515088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:44.536021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:44.537868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:44.638585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:44.647081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 515/1010] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:44.662593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 496/1011] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:44.674139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.686511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.783505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:44.827268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13f5f5-92c3-4210-8392-30d5a799d0a2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:44.864996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2b1b-e5c4-4273-b3bf-62d41f4c90ab None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:44.883404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:44.884631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.885310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.885734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.887155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:44.888832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:44.905472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:44.906141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:44.906460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:44.908300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:44.910772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:44.913684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.913684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.913684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:44.914336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 516/1012] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:44.914601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:44.914846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.932009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 497/1013] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:44 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:45.051865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ea57d-c654-45ad-b922-aac84f68a308 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:45.103973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:45.104663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.104944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.105767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.106541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:45.106849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:45.108267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:45.110840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:45.120286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 517/1014] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:45.122234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:45.198653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:45.263645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1473-9504-4074-8010-2ae89452fe31 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:45.308740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:45.309318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.309598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.309881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.310643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:45.310953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:45.312377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9de556-e5ef-4190-bfe5-31d5aed515f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:45.318849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 498/1015] 10.222.0.77 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:45.325465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:45.501175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a165fb-e74b-4ba4-99ac-b98ccbe013bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:45.543108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:45.543776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.544095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.544411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.545094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:45.546615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe2e23-6982-4018-813d-b04f4a655e6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:45.578551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 518/1016] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:45.999498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed196-f045-4e62-a57a-b3981583f7ba None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:46.000335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed196-f045-4e62-a57a-b3981583f7ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:46.000733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed196-f045-4e62-a57a-b3981583f7ba None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:46.003798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 499/1017] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:46.014121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:46.014652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:46.015144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:46.043981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:46.046075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:46.161745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:46.194514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 519/1018] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:46.234559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aed196-f045-4e62-a57a-b3981583f7ba None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:46.437085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aed196-f045-4e62-a57a-b3981583f7ba None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:46.470438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:46.504972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:46.506251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:46.506768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:46.507257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:46.509298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:46.511814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb878b-96d4-4600-ad33-57058f91c99a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:46.515091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 500/1019] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:46.522759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:46.795754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446277b-3357-48fc-b767-168aeac8c0f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:46.867471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:46.868615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:46.869261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:46.869700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:46.871707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:46.874710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c6b51-7b31-444a-b4c0-bb87d92d7326 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:46.917361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 520/1020] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:46 2022] POST /identity/v3/projects => generated 311 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:47.348865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a03e-b7c8-4b8a-87ca-4ed301cb0135 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:47.349421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a03e-b7c8-4b8a-87ca-4ed301cb0135 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:47.349767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a03e-b7c8-4b8a-87ca-4ed301cb0135 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:47.352888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 501/1021] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:47.362442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:47.362944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:47.363416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:47.393050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:47.395203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:47.448311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:47.505879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 521/1022] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:47.547426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a03e-b7c8-4b8a-87ca-4ed301cb0135 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:47.732426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a03e-b7c8-4b8a-87ca-4ed301cb0135 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:47.778926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:47.779745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:47.780031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:47.780281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:47.780959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:47.782444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-406dca67-9e14-4cc7-96bf-d37c2b9c6c2f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:47.795733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:47.797692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:47.799997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:47.802328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 502/1023] 10.222.0.77 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:47 2022] GET /identity/v3/services/image => generated 86 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:47.803834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:47.937812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd09ad6-e653-4b4d-92b4-91a26ffe06ed None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.001769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.002803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.003209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.003598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.004716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] RBAC: Adding query filter params (name=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:48.005168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.007446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.021871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 522/1024] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:48.022097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c06f-7829-45af-86c8-6a22613aad5e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.022206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c06f-7829-45af-86c8-6a22613aad5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.023271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91c06f-7829-45af-86c8-6a22613aad5e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.023785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.025869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 503/1025] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:48.033720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.034011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.034348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.058724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.059016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:48.060812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:48.119961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.170789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 504/1026] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:48.184395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e77c-db1f-4eeb-94fe-00ad1ff767cc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.192639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.241331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.242453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.242881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.243345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.244656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] RBAC: Adding query filter params (type=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:48.245189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.248435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.262457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 523/1027] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:48.268875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.385248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6349d3-ad4c-4b08-a3cc-7c1843d45231 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.414077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.427284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5ed4-311e-42ff-bd95-8bcb0d1f5863 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.459707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.461049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.461559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.462062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.479225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.481066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.481561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.482045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.483887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] RBAC: Adding query filter params (interface=public, service_id=60ba364425764d26a8116cac326f6ad7, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:48.484595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.484961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:48.487468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.493778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 524/1028] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/endpoints?service_id=60ba364425764d26a8116cac326f6ad7&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:48.499806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:48.500758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:48.501739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:48.502661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:48.503346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 505/1029] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:48.511685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.570963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.655155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e779-b99c-4091-b7c6-e070ef3fd8ab None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.666476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef2b0-d93c-41ce-85c6-93ff19bf70cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.699429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.700130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.700412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.700687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.701389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.702922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.711489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 525/1030] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/services => generated 1957 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:25:48.713686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.715033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.715434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.715859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.716659 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:48.717020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.718676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.730473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 506/1031] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:48.736578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.766662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.874610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9d93d-b642-4d8d-abd7-cfd737548b99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.928834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.929992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.930517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.930988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.947473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.950712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:48.985237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:48.987359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:48.989568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:48.989568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:48.989568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:48.989568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:48.989568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 526/1032] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:48.997721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.204412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:49.209999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59df50-68f5-40ee-8e67-a1dbf5d0c17d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:49.276065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:49.276924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.277331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.277805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.279221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:49.279674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:49.281799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc44695-13f0-4258-90e1-fa02c4ec8ac0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:49.299590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 507/1033] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:49.307388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.412577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:49.451352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb39b5a-3a4c-45aa-a5a7-2a7760964006 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:49.464876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9511e6-4ba1-423e-87ae-0db1b832f18e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.465448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9511e6-4ba1-423e-87ae-0db1b832f18e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.466068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9511e6-4ba1-423e-87ae-0db1b832f18e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.469113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 508/1034] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:49.478209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.478751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.479241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.494445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:49.495322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.496175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.496503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.505487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:49.507020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:49.510052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:49.512271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:49.524703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:49.525745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:49.526846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:49.533491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 527/1035] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:49.535037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.639646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 509/1036] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:49.665443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.679351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210736d-d3a1-473a-8ccd-5e695f396ade None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:49.722579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:49.729137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:49.729829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.730798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.731084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.731959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:49.732276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:49.733765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:49.757285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 528/1037] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:49.763415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.794759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:49.810349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae80e0f-4316-44b3-9d61-763dee7f0def None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:49.864198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:49.865259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.865780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.866228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:49.868025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:49.869836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:49.878139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 510/1038] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:49.879410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:49.909566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15345cc-83d9-4216-b751-f2370d4c1d02 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:49.952983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:49.953904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:49.954189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:49.954495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.019130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd73f85-fe3e-4014-913b-e7b0415af262 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:50.029922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:50.047700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:50.049115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:50.070935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:50.072095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.072644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.073122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.074663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:50.076788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:50.110704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 511/1039] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:25:49 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:50.186798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 529/1040] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:50.193218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:50.213969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:50.385137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c13582-1cb7-4461-966a-96cdc3bb29c6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:50.448556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:50.449953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.450236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.450546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.465400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:50.469538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:50.493888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:50.494936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:50.496376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:50.496376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:50.496376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:50.496376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:50.498959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 512/1041] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:50.508574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:50.631159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:50.644781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561d416-e81a-4331-8ca4-08ed595b5a3b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:50.690283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:50.691451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.691887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.692313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.693248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:50.693916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:50.695642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:50.716829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 530/1042] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:50.718814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:50.822866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:50.869157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fa5e-4707-48de-83d4-3effea84c2d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:50.914133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:50.914978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.915254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.915521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.941625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:50.943183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-88d9c7dd-4959-4c19-b1b2-9122d9a60169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:50.960160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:50.961144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:50.962687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 513/1043] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:50.969310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.048968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1bb2-e480-4e7a-b4bf-bcb418f5e7ab None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.049241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1bb2-e480-4e7a-b4bf-bcb418f5e7ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.049528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1bb2-e480-4e7a-b4bf-bcb418f5e7ab None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.051320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 514/1044] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:51.059919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.060373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.060814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.088782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:51.090903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:51.100567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2fd4-b2f8-430d-94cb-70d0033873a2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.153276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.154266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.154266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.154604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.155532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:51.155951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.157527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.178517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 531/1045] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:51.181036 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.184981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 515/1046] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:51.196811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.208077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.282891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.318183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27c115-b264-4392-a790-9a9c6c461c10 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.350810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd77af-4fa2-4306-bb90-b34c5b6c5306 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.369655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.370827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.371318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.371801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.373379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] RBAC: Adding query filter params (role.id=1a3770e02a3e474e938ba08f11c36fbe, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:51.373895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.376629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.384484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 532/1047] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=1a3770e02a3e474e938ba08f11c36fbe => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:51.397198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.398299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.398795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.399238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.400081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.402309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:51.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:51.417235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:51.418752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:51.419525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 516/1048] 10.222.0.77 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:51.427421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.521272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.553826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a483f0-16e8-49b8-9fcd-3768d9a09c9c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.601379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.602089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.602369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.602688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.603428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] RBAC: Adding query filter params (name=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:51.603822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.605231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.611786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 533/1049] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:51.617834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.752095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312950b-73ce-402a-9495-286e5464d670 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.777633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.796738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.797417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.797701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.797975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.798758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] RBAC: Adding query filter params (type=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:51.799063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.800485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12682213-6118-415a-a64f-3c13fd6675af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.806826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 517/1050] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:51.813294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:52.000592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e42e1-867b-4117-8b87-efef0d184a88 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:52.065056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:52.066062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:52.066527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:52.068556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:52.068556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:52.070281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d890a-62d0-423b-98d2-b1333617c25f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:52.084113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:52.110446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 534/1051] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:52.643750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf72-064c-4fb0-967c-9f96fee1bef2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:52.643750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf72-064c-4fb0-967c-9f96fee1bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:52.643750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf72-064c-4fb0-967c-9f96fee1bef2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:52.645162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 518/1052] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:52.654118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:52.654674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:52.655171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:52.685528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:52.687885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:52.786734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 535/1053] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:52.810714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550caf72-064c-4fb0-967c-9f96fee1bef2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:52.927575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550caf72-064c-4fb0-967c-9f96fee1bef2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:52.983740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:52.984540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:52.984817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:52.985087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:52.998129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:53.006231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:53.007372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:53.008303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:53.010486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 519/1054] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:53.014213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:53.127436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b57c-07a4-4328-adb2-cde31c236861 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:53.151029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:53.169934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:53.170660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.170946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.171227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.172022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:53.172314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:53.173757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:53.184237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 536/1055] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:53.190242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:53.243495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:53.306085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b313f-993c-4195-9276-3cd1cb53c5c5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:53.344972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:53.345875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.346153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.346525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] PATH_INFO: `/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.363124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] RBAC: Authorizing `identity:get_user(user_id=6cf7dcb409fd4b95bf87bcbd7556c9a6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:53.364635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:53.366787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 520/1056] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:53.373466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:53.494531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4d3f-df2d-4f8c-85a3-3768ce665ceb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:53.531892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:53.532569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.532851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.533126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.596335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:53.621309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:53.622854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fcaf-ddc7-45f1-b646-1cf8a7e56822 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:53.805723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 537/1057] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:53.812219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.016333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854e86-d344-46dd-b498-d56aa03f671a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.020417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.058593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526e35-4488-478d-86e9-7c1e5a006726 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.059333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526e35-4488-478d-86e9-7c1e5a006726 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.059333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526e35-4488-478d-86e9-7c1e5a006726 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.060710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 538/1058] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:54.066513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.066801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.067049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.092061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.092954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.093132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:54.093288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.093565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.094869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.095156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:54.096341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.098498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 521/1059] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:54.104983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.227164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 539/1060] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:54.264671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.275365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a269-3985-4e29-95b3-46785e395cb5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.288299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.340230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.341259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.341553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.341957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.343509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=958573196b3a4887a5870dd56f9b306c, user.id=6cf7dcb409fd4b95bf87bcbd7556c9a6) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:54.344038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.346644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.356259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 522/1061] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/role_assignments?user.id=6cf7dcb409fd4b95bf87bcbd7556c9a6&scope.project.id=958573196b3a4887a5870dd56f9b306c&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:54.430384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cbf29-3efe-4462-804f-3987601fc3b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.476632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.477473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.477748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.478014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.492374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:54.501456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:54.512145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:54.514276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 540/1062] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:54.518990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.584247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.720611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4075e7-ac2c-4e7a-ac8a-627da1dcef80 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.787557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.788662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.789149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.789639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.791129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:54.791683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.795120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.818448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 523/1063] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:54.823320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.963486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:55.030152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3a54b-d0a9-428f-9d8e-20fadce2dc59 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:55.095723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:55.097111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.097595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.098100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.119333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:55.122047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-42f6aff6-5d52-465f-b006-943bbd737692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:55.153856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:55.154880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:55.155800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:55.155800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:55.155800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:55.155800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:55.156668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 541/1064] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:55.164967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:55.207656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1eaf-2284-4cad-a12f-90353a546147 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.208156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1eaf-2284-4cad-a12f-90353a546147 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.208488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1eaf-2284-4cad-a12f-90353a546147 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.210995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 542/1065] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:55.220399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072490-383b-4112-8781-eaa16b56458f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.221055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072490-383b-4112-8781-eaa16b56458f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.221906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072490-383b-4112-8781-eaa16b56458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.244918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-82072490-383b-4112-8781-eaa16b56458f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:55.246313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-82072490-383b-4112-8781-eaa16b56458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:55.328817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:55.337533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 543/1066] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:55.341496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c6f57-8766-443c-bee8-039a6189a8fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:55.376308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82072490-383b-4112-8781-eaa16b56458f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:55.409163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:55.410292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.410832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.411324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.412678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:55.413406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:55.415881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:55.439475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 524/1067] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:55.447428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:55.477382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:55.541639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82072490-383b-4112-8781-eaa16b56458f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:55.593204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5be87-226b-42af-98db-178481d5263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:55.594307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5be87-226b-42af-98db-178481d5263e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.594840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5be87-226b-42af-98db-178481d5263e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.595322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5be87-226b-42af-98db-178481d5263e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.657718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0a4e7-52f0-4c1e-a067-3122962c3f09 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:55.706929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:55.707746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.708037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.708311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.717498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:55.719055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:55.734529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5be87-226b-42af-98db-178481d5263e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:55.736308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:55.737850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:55.740509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:55.744118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5be87-226b-42af-98db-178481d5263e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:55.744347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 525/1068] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:55.744347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:55.767314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:55.848490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:55.909206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786606b-a17c-4e4c-94bf-77cd366ff00a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:55.939632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 544/1069] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:55.950690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5be87-226b-42af-98db-178481d5263e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:55.960404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:55.961423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:55.961916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:55.962387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:55.963507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:55.964064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:55.966637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:55.996107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 526/1070] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:56.002547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.124987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.140084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5be87-226b-42af-98db-178481d5263e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.174744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f12d8-6314-4603-b233-2eae4bf22c6a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.210377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.211974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.212449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.212925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.223614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.224614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.225021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.225422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.233839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.236603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.239964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.241984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:56.263717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:56.265473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:56.267239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:56.267239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:56.267239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:56.267239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:56.270940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 527/1071] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:56.271861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:56.275011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:56.278515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:56.278515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:56.278515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:56.278515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:56.279194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.281947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 545/1072] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:56.283354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.303037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.421249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390482a-490c-4c5b-961e-0ec616830b2e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.455889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.472829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.474151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.474703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.475184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.476417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:56.477454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.479480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.482695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400487b-5b55-4532-876f-507ab24af152 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.501285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 528/1073] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:56.503660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.537625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.538626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.539086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.539584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.540777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:56.541240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.543609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d3c0-88f3-4f54-bb4a-4b77b94916bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.558029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 546/1074] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:56.732904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1f46d-6ff0-4821-b1d0-c2cd86945071 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.743193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b6d1-f73f-4c77-8e15-d7c39257093f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.743655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b6d1-f73f-4c77-8e15-d7c39257093f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.744069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b6d1-f73f-4c77-8e15-d7c39257093f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.746947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 547/1075] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:56.754180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.755066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.755506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.786947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:56.788887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:56.798288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.799639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.800101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.800555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.812968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.815974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:56.836949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:56.837998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:25:56.841545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 529/1076] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:56.850234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.906205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.919655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 548/1077] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:56.926959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.948291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.015993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f012-de94-4f6a-adf9-69de652daaa6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.065339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.066647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.067031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.067483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.069189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:57.070914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.073989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.099031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 530/1078] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:57.109065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.112211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962d079-7173-4f88-8b98-4afc15213c10 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.154836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.156243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.156747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.157230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.176303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:57.185170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:57.187318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:57.189380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 549/1079] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:57.194524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.259876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff631d4-8115-438c-9b0a-5ac141d3d66b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.263014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:57.273053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:57.312843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.314045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.314443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.314818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/1a3770e02a3e474e938ba08f11c36fbe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.316997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020a0e-07e8-44ac-83c0-769bb8f2116e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.317800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=bd1cd1cd83fa43cab0dcbd1698c715fd, role_id=1a3770e02a3e474e938ba08f11c36fbe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.319850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.339190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 531/1080] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:57 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/bd1cd1cd83fa43cab0dcbd1698c715fd/roles/1a3770e02a3e474e938ba08f11c36fbe => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:25:57.362982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.363828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.364217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.364584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.365498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:57.365913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.367562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.379098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 550/1081] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:57.387830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.550932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a374a-f0f4-41c1-ae3c-804dcacebc70 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.596401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.597282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.597621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.598028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] PATH_INFO: `/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.599818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] RBAC: Authorizing `identity:get_user(user_id=6cf7dcb409fd4b95bf87bcbd7556c9a6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.601364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.603217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 532/1082] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:57 2022] GET /identity/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6 => generated 285 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:57.612937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.767817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ab18-62ff-4a16-bd06-4b89f9c3f2de None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.788220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:57.809037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.809998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.810282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.810596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.812407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.813893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9a6a9-a997-4752-9f29-2d7a1cd89685 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.815769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 551/1083] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:57 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:57.822951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.865917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:57.950200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f98af-99a3-4b96-a443-87a7bd0a4423 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.991864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.992958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.993370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.993787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c/users/6cf7dcb409fd4b95bf87bcbd7556c9a6/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.996411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=958573196b3a4887a5870dd56f9b306c, user_id=6cf7dcb409fd4b95bf87bcbd7556c9a6, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.998086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9124dae-034d-4b3f-ba7b-12dcb5ba7fe6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:58.018097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 533/1084] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:57 2022] PUT /identity/v3/projects/958573196b3a4887a5870dd56f9b306c/users/6cf7dcb409fd4b95bf87bcbd7556c9a6/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:25:59.447620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bf74-5744-4894-9f31-5ff26a2ea19c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.448198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bf74-5744-4894-9f31-5ff26a2ea19c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.448198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bf74-5744-4894-9f31-5ff26a2ea19c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.449641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 552/1085] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:59.456667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.457268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.457640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.485947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:59.487411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:59.567908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 534/1086] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:59.592282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252bf74-5744-4894-9f31-5ff26a2ea19c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:59.773773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252bf74-5744-4894-9f31-5ff26a2ea19c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:59.838140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:59.839645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.840138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.840621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.860289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c2034ea6-7bb4-4c28-ae77-30686da5867a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:59.876623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:59.877732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:59.878668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:25:59.879803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 553/1087] 10.222.0.77 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:59.888554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:59.918750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.063243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0d3a6-a59e-43ed-8a63-3d58372b1fe7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.129646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fac461-690f-44bb-8f58-40484f44ec76 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.129937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fac461-690f-44bb-8f58-40484f44ec76 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.130352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fac461-690f-44bb-8f58-40484f44ec76 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.130546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.131727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.131840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 554/1088] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:00.132221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.132710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.134099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:00.134676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.137348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.137565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.137861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.138144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.158469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 535/1089] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:00.158723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:00.158723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:00.161136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.239747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.258688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.276997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accc5e6-50ea-4fea-a38b-32f546f6cec3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.285898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 555/1090] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:00.311806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.317739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.318503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.318790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.319068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.333202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.334732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:26:00.354185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:00.356582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:00.357521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:26:00.357521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:26:00.357521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:26:00.357521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:00.357521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 536/1091] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:00.365755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.458752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9314-87cf-4938-80f0-bbe3626568c7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.502610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.503646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.504041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.504525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.514614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f674-27d0-4cf3-866e-730531c988cf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.525784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:00.534380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:00.536115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:00.537764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:26:00.541053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 556/1092] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:00.542049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.559911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.560593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.560885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.561149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.561893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:00.562198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.563654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.572102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.580744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 537/1093] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:00.588167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.605356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.700660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0b574-a7a1-4030-97fc-1af3558fa91d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.740364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.741073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.741345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.741623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.742450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:00.742749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.744195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.746041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab36b7-6671-4ddf-9d5e-7dec8558667b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.760586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 557/1094] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:00.762904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.801493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.802319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.802674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.802908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.813505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.816369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:26:00.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:00.848189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:00.850045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 538/1095] 10.222.0.77 () {56 vars in 1093 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:00.855975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:00.889026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.894310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c2a54-cdc0-468f-ad44-f8202a1308ef None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:00.943383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:00.946713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:00.947828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:00.948878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:00.949259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] PATH_INFO: `/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:00.951490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] RBAC: Authorizing `identity:get_user(user_id=6cf7dcb409fd4b95bf87bcbd7556c9a6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:00.953110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:00.955316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 558/1096] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/users/6cf7dcb409fd4b95bf87bcbd7556c9a6 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:00.962194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.005330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb58a1-ac3d-4b16-b624-a3006c48ae73 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.056249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.056890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.057467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.058037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.059621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:01.060157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.062733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.091292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 539/1097] 10.222.0.77 () {56 vars in 1133 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:01.098280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.110691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55bd0-6946-46f1-8327-72152e43fdb1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.166424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.168109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.168589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.169062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.218831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd89dc7-4491-4717-8447-deaedb6acd56 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.229548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.243670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.270574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.271424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.271737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.272478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.280794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.282621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.364669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.366242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.419494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 559/1098] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:26:01.425897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.519714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.525805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 540/1099] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:26:01.527621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.557413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.574715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b440020-8cfb-4774-9a0f-0fbba549e6a2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.624376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.625279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.625708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.626121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.627691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.629487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.631534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 560/1100] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:01.639384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.671635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd66b3-c515-45c4-8cb3-233c32ac81f9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.717643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.718746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.719126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.719499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.733133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.735122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:26:01.756423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:01.758132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:01.759635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:26:01.759635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:26:01.759635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:26:01.759635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:01.759635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 541/1101] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:01.768303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:01.770575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8fcb-3d81-420c-a8fc-a2a73aadba16 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.820895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.821713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.822092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.822521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.824166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=958573196b3a4887a5870dd56f9b306c, user.id=6cf7dcb409fd4b95bf87bcbd7556c9a6) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:01.824592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.826255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.828998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.837970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 561/1102] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/role_assignments?user.id=6cf7dcb409fd4b95bf87bcbd7556c9a6&scope.project.id=958573196b3a4887a5870dd56f9b306c&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:01.862181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.907962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03051127-c5ea-4f95-b23c-3ec0d06c82e5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:01.954289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:01.955199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.955585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.955965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.956880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:01.957343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:01.958985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:01.973005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 542/1103] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:01.980492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.132261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d21d51-83b5-4aef-b114-5f7643a58aa9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.173774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.174637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.174930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.175209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.184452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.186047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:26:02.202528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:02.204816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:26:02.206932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 562/1104] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:02.211060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.327355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552c9e-447c-4bb3-a39b-8c23d96cf815 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.359051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.368161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.368845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.369127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.369401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.370807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:02.371119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.372622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2518f9-10c3-473d-890e-25ecfa665e7d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.386346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 543/1105] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:02.392994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.459761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.510712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9885c-85ef-4122-b6e5-854fbdf9841b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.554389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.555441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.555441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.555740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.556592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] RBAC: Adding query filter params (role.id=1a3770e02a3e474e938ba08f11c36fbe, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=bd1cd1cd83fa43cab0dcbd1698c715fd) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:02.556902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.558928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3f525-5023-4373-9871-f47ca03732b3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.566521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 563/1106] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/role_assignments?user.id=bd1cd1cd83fa43cab0dcbd1698c715fd&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=1a3770e02a3e474e938ba08f11c36fbe => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:04.152689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960706-6364-497f-9d6a-9a4793a74d31 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:04.153279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960706-6364-497f-9d6a-9a4793a74d31 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:04.153279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960706-6364-497f-9d6a-9a4793a74d31 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:04.154714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 544/1107] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:04.161020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:04.161670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:04.161670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:04.180379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:04.181674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:04.262597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 564/1108] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:04.287731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38960706-6364-497f-9d6a-9a4793a74d31 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:04.414247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38960706-6364-497f-9d6a-9a4793a74d31 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:04.455748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:04.456554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:04.456837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:04.457109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:04.458385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:04.459879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a304d08-6b7b-438b-920b-c563aebdb485 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:04.461633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 545/1109] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:26:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:04.471912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:04.551597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:04.608772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88941340-c73c-4432-ba2a-d3f6ff37bf80 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:04.649407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:04.650103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:04.650388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:04.650698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:04.651586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:04.653070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee7998-f994-442d-9bd7-d8cea9d8e016 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:04.674894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 565/1110] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:26:04 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:06.868123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007acee1-d9b0-40c7-b419-c26fb5b1bf9f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:06.868690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007acee1-d9b0-40c7-b419-c26fb5b1bf9f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:06.868690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007acee1-d9b0-40c7-b419-c26fb5b1bf9f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:06.870660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 546/1111] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:06.877836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:06.878375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:06.878862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:06.908000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:06.910024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:07.019683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 566/1112] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:07.045059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007acee1-d9b0-40c7-b419-c26fb5b1bf9f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:07.109343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:07.189863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007acee1-d9b0-40c7-b419-c26fb5b1bf9f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:07.240648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:07.242111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:07.242650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:07.243132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] PATH_INFO: `/v3/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:07.245842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] RBAC: Authorizing `identity:get_role(role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:07.248411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:07.251083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 547/1113] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:07 2022] GET /identity/v3/roles/345ac5f37e244b4faa681fbd95404a3d => generated 223 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:07.257324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:07.302983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:07.373011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae36e-0c12-417d-a6dc-f3ac3fb39315 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:07.414231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:07.415193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:07.415478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:07.415755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] PATH_INFO: `/v3/users/e2447304c9b54df79eb7972b7d96b27f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:07.431519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] RBAC: Authorizing `identity:get_user(user_id=e2447304c9b54df79eb7972b7d96b27f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:07.433029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:07.434871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 567/1114] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:07 2022] GET /identity/v3/users/e2447304c9b54df79eb7972b7d96b27f => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:07.441636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:07.557843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40d0b-507a-4cb8-b03e-d62cf32a2add None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:07.586888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:07.604386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:07.605067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:07.605353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:07.605632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:07.687031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:07.688573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:07.806792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 548/1115] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:07.813794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:07.927539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:07.935395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac32004-da33-4988-ad00-31e142763379 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:07.976380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:07.977655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:07.978045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:07.978441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:07.980639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:07.982715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606540fc-5e02-41ee-bc15-533f776ecd33 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:07.984573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 568/1116] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:07 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:07.991311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:08.020261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:08.106920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6bea-ec68-4040-85f5-8b271d49b8ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:08.145991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:08.146696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.146979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.147254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.148117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=958573196b3a4887a5870dd56f9b306c, user.id=e2447304c9b54df79eb7972b7d96b27f) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:08.148429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:08.149925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887d48a-4d5f-4c84-b9b8-48ac19e53401 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:08.156655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 549/1117] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:07 2022] GET /identity/v3/role_assignments?user.id=e2447304c9b54df79eb7972b7d96b27f&scope.project.id=958573196b3a4887a5870dd56f9b306c&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:10.263545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7547558-76dc-4370-9c7b-18d8335db5ed None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:10.264098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7547558-76dc-4370-9c7b-18d8335db5ed None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:10.264249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7547558-76dc-4370-9c7b-18d8335db5ed None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:10.265624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 569/1118] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:10.272609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:10.273139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:10.273561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:10.302621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:10.304717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:10.428810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 550/1119] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:10.452375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7547558-76dc-4370-9c7b-18d8335db5ed None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:10.566189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7547558-76dc-4370-9c7b-18d8335db5ed None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:10.604656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:10.605523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:10.605803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:10.606076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] PATH_INFO: `/v3/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:10.607625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] RBAC: Authorizing `identity:get_role(role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:10.609014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:10.610431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 570/1120] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:10 2022] GET /identity/v3/roles/345ac5f37e244b4faa681fbd95404a3d => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:10.617039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:10.648306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:10.747078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478639c-175d-4226-8ad3-7fe8cf692ea6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:10.785205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:10.786084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:10.786356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:10.786656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] PATH_INFO: `/v3/users/e2447304c9b54df79eb7972b7d96b27f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:10.787970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2447304c9b54df79eb7972b7d96b27f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:10.789425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:10.791147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 551/1121] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:10 2022] GET /identity/v3/users/e2447304c9b54df79eb7972b7d96b27f => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:10.801052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:10.838088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:10.921459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20483-516b-440b-bf24-fa9d873fa67e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:10.960078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:10.960957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:10.961241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:10.961519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:10.962881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:10.964889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546fbbd-2544-4ef8-9f95-dd4e36b9c7b6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:10.966735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 571/1122] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:10 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:10.973130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:11.070251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:11.093044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d9d9e-820d-4503-b8e3-591992a9c5b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:11.152029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:11.152936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:11.153217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:11.153494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c/users/e2447304c9b54df79eb7972b7d96b27f/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:11.155952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] RBAC: Authorizing `identity:create_grant(project_id=958573196b3a4887a5870dd56f9b306c, user_id=e2447304c9b54df79eb7972b7d96b27f, role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:11.157450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e98170-5a46-427f-9c23-5e6c8ca1e3fc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:11.172385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 552/1123] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:26:10 2022] PUT /identity/v3/projects/958573196b3a4887a5870dd56f9b306c/users/e2447304c9b54df79eb7972b7d96b27f/roles/345ac5f37e244b4faa681fbd95404a3d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:26:13.386454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b18-a946-4a41-b8f8-ab1e2ea1633b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:13.387301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b18-a946-4a41-b8f8-ab1e2ea1633b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:13.387301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b18-a946-4a41-b8f8-ab1e2ea1633b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:13.388654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 572/1124] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:13.394798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:13.395341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:13.395502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:13.413739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:13.415081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:13.492773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 553/1125] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:13.516717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc04b18-a946-4a41-b8f8-ab1e2ea1633b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:13.631249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:13.636220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc04b18-a946-4a41-b8f8-ab1e2ea1633b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:13.685894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:13.687367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:13.687790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:13.688249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] PATH_INFO: `/v3/roles/345ac5f37e244b4faa681fbd95404a3d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:13.703906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] RBAC: Authorizing `identity:get_role(role_id=345ac5f37e244b4faa681fbd95404a3d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:13.706034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:13.708802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 573/1126] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:13 2022] GET /identity/v3/roles/345ac5f37e244b4faa681fbd95404a3d => generated 223 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:13.715900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:13.805343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:13.833367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c190f-dc7d-47d2-91c5-72f91ffe200f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:13.883986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:13.885573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:13.886058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:13.886594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] PATH_INFO: `/v3/users/e2447304c9b54df79eb7972b7d96b27f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:13.888818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] RBAC: Authorizing `identity:get_user(user_id=e2447304c9b54df79eb7972b7d96b27f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:13.891577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:13.894942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 554/1127] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:13 2022] GET /identity/v3/users/e2447304c9b54df79eb7972b7d96b27f => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:13.901573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:14.019854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d5ad4-38b8-4725-89e5-022d206cd087 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:14.060574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:14.061257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.061544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.061824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.106645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:14.148038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:14.149529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:14.269770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 574/1128] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:14.276904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:14.397218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d735-a51e-4a7a-bfe4-5d1629cd1a00 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:14.436364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:14.437250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.437535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.437808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] PATH_INFO: `/v3/projects/958573196b3a4887a5870dd56f9b306c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.439125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] RBAC: Authorizing `identity:get_project(project_id=958573196b3a4887a5870dd56f9b306c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:14.440591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264a5d3-19fe-4c52-876c-ecd7da7de05c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:14.442370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 555/1129] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:14 2022] GET /identity/v3/projects/958573196b3a4887a5870dd56f9b306c => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:14.450054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:14.564501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:14.623629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d5d5e-00be-4e36-91c1-6cec0db72c61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:14.681750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:14.683379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.683835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.684290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.685901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] RBAC: Adding query filter params (role.id=345ac5f37e244b4faa681fbd95404a3d, scope.project.id=958573196b3a4887a5870dd56f9b306c, user.id=e2447304c9b54df79eb7972b7d96b27f) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:14.686445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:14.689132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2f4a1-a0a4-4a3c-abf5-1358ce6b5027 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:14.700787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 575/1130] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:14 2022] GET /identity/v3/role_assignments?user.id=e2447304c9b54df79eb7972b7d96b27f&scope.project.id=958573196b3a4887a5870dd56f9b306c&role.id=345ac5f37e244b4faa681fbd95404a3d => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:16.818174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a60a8-2717-4dee-8488-3eee029c6bb3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.818850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a60a8-2717-4dee-8488-3eee029c6bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.818850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a60a8-2717-4dee-8488-3eee029c6bb3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.820152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 556/1131] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:16.826087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.828691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.828691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.846650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:16.848894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:16.927348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 576/1132] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:16.951469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a60a8-2717-4dee-8488-3eee029c6bb3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:16.968598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:17.068397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a60a8-2717-4dee-8488-3eee029c6bb3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:17.107883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:17.108690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:17.108963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:17.109230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:17.110570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:17.112032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4e90a-5f00-4903-8582-aa87bd0ef43e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:17.113743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 557/1133] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:26:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:17.120010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:17.189127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:17.233385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaad9a-1e60-445e-b157-ed982bf52b81 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:17.270698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:17.271396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:17.271675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:17.271949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:17.272689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:17.274178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe10aa6-1f52-4031-8739-b79d390d5f02 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:17.303412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 577/1134] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:26:17 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:26:19.471868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9d92-8707-4af5-8a20-a2169f1b34e4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.472434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9d92-8707-4af5-8a20-a2169f1b34e4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.472434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c9d92-8707-4af5-8a20-a2169f1b34e4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.473907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 558/1135] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:19.480420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.480974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.480974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.500130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:19.501548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:19.581019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 578/1136] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:19.607452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9d92-8707-4af5-8a20-a2169f1b34e4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:19.702498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:19.726329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c9d92-8707-4af5-8a20-a2169f1b34e4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:19.764526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:19.765415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.765741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.766061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.767601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:19.769233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bfddb-9c72-4c81-a446-075c711c9ac5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:19.771095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 559/1137] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:26:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:19.778365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:19.831304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:19.892551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102c07-4acc-444a-b3d3-59baf3523e88 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:19.932002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:19.932686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.932966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.933238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.933968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:19.935445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb07297-fdb0-426b-a383-2b3ffa420b6f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:19.958541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 579/1138] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:26:19 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:26:22.107061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e993fac-4e9f-42ee-ab2e-59605e01cb3a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.107602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e993fac-4e9f-42ee-ab2e-59605e01cb3a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.107602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e993fac-4e9f-42ee-ab2e-59605e01cb3a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.108961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 560/1139] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:22.114593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.115154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.115154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.132498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:22.133719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:22.246779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 580/1140] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:22.272589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e993fac-4e9f-42ee-ab2e-59605e01cb3a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.390150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e993fac-4e9f-42ee-ab2e-59605e01cb3a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.429127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.429933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.430215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.430519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.442132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:22.450832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:22.452726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:22.454763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 561/1141] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:22.458819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.494575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.575895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab68d95-6244-4d05-8542-ac193ba870f0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.614934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.615627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.615913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.616190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.616965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:22.617272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.618724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.629018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 581/1142] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:22.638662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.654181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.757131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e62e4d-e681-4133-9e07-5e7cff97b2de None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.799131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb985-513d-4415-a557-7af495954697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.800023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb985-513d-4415-a557-7af495954697 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.800305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb985-513d-4415-a557-7af495954697 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.800580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb985-513d-4415-a557-7af495954697 None admin] PATH_INFO: `/v3/users/9e173bb6886946ffb7064036601974e4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.816806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb985-513d-4415-a557-7af495954697 None admin] RBAC: Authorizing `identity:get_user(user_id=9e173bb6886946ffb7064036601974e4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.818294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb985-513d-4415-a557-7af495954697 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.820230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 562/1143] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/users/9e173bb6886946ffb7064036601974e4 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:22.830328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.928197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.943289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55fb0-9dad-4f09-95a9-a8e94062e72a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.983167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.983819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.984109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.984387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:23.100417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:23.101898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:23.205511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:23.221562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 582/1144] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:23.228036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb985-513d-4415-a557-7af495954697 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:23.304611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:23.368157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb985-513d-4415-a557-7af495954697 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:23.433112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:23.434775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:23.435267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:23.435745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] PATH_INFO: `/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:23.438094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] RBAC: Authorizing `identity:get_project(project_id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:23.440893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb00af-70ff-4e79-b9a7-d57072e1ff40 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:23.444250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 563/1145] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:23 2022] GET /identity/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5 => generated 311 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:23.450848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:23.562969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ca3fa-cafa-4d9d-9663-e6c231969d4b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:23.600851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:23.601512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:23.601794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:23.602066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:23.602939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5, user.id=9e173bb6886946ffb7064036601974e4) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:23.603251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:23.605290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91603-9316-44eb-8c2b-f6a88f74c295 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:23.611501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 583/1146] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:23 2022] GET /identity/v3/role_assignments?user.id=9e173bb6886946ffb7064036601974e4&scope.project.id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:25.678534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce347-67fc-4f08-a83d-91e704d2a5fb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.679082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce347-67fc-4f08-a83d-91e704d2a5fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.679082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce347-67fc-4f08-a83d-91e704d2a5fb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.680501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 564/1147] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:25.686539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.687402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.687402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.705599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:25.706844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:25.763907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:25.784661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 584/1148] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:25.809465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bce347-67fc-4f08-a83d-91e704d2a5fb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.930163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bce347-67fc-4f08-a83d-91e704d2a5fb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:25.969868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:25.970707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.971070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.971350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.973588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:25.984498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:25.992426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:25.993757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:25.994959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 565/1149] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:26.003087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:26.112216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b8db-af37-414f-ab97-fa863b72e57b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:26.148902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:26.149589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:26.149874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:26.150150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:26.151521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:26.151826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:26.153224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:26.163191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 585/1150] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:26.169528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:26.282568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81397-8ffe-4aef-a100-17b17c1c6187 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:26.317965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:26.318875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:26.319151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:26.319416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] PATH_INFO: `/v3/users/9e173bb6886946ffb7064036601974e4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:26.320689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] RBAC: Authorizing `identity:get_user(user_id=9e173bb6886946ffb7064036601974e4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:26.322153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:26.324460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 566/1151] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:26 2022] GET /identity/v3/users/9e173bb6886946ffb7064036601974e4 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:26.330989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:26.388471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:26.445670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2604-ed86-4ea8-9efb-7f84c4d0e538 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:26.484795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:26.485673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:26.485955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:26.486231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] PATH_INFO: `/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:26.487536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] RBAC: Authorizing `identity:get_project(project_id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:26.489002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc428a-b71a-4760-9aaf-523c0839386e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:26.490821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 586/1152] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:26 2022] GET /identity/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:26.498034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:26.569793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:26.618318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce763a-1870-42b9-8548-4888b126bca7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:26.656627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:26.657530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:26.657805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:26.658073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] PATH_INFO: `/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5/users/9e173bb6886946ffb7064036601974e4/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:26.660238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5, user_id=9e173bb6886946ffb7064036601974e4, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:26.661723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eaef7-f2fb-47bd-b421-87aa3ea25fd2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:26.681572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 567/1153] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:26:26 2022] PUT /identity/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5/users/9e173bb6886946ffb7064036601974e4/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:26:28.828583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b6cc-81ce-4a3f-8b37-529a959982d5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.829148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b6cc-81ce-4a3f-8b37-529a959982d5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.829148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1b6cc-81ce-4a3f-8b37-529a959982d5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.830670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 587/1154] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:28.836614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.837134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.837243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.855224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:28.856456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:28.931427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 568/1155] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:28.956595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1b6cc-81ce-4a3f-8b37-529a959982d5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.074889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1b6cc-81ce-4a3f-8b37-529a959982d5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.093398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.114391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.115232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.115517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.115796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.129611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:29.137691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:29.138923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:26:29.139950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 588/1156] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:29.145603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.206685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.260916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aed77-9b1d-45a7-baa0-141bac12ade9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.298789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.299607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.299935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.300333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.301199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:29.301557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.303053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.313080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 569/1157] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:29.319176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.439129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069c21-1115-4208-a9a9-a2ffbe7a0514 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.490800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.492255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.492761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.493243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] PATH_INFO: `/v3/users/9e173bb6886946ffb7064036601974e4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.495571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] RBAC: Authorizing `identity:get_user(user_id=9e173bb6886946ffb7064036601974e4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.498305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.501452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 589/1158] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/users/9e173bb6886946ffb7064036601974e4 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:29.507536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.626126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9f56-d291-46a9-ab99-d514a8288b74 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.641605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.665318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.666753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.667040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.667313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.742361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.743859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.857778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 570/1159] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:29.865000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.954683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:30.051203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b80811-db97-4d09-a19f-3f2a91a7b5ed None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:30.114090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:30.115621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:30.116151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:30.116685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] PATH_INFO: `/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:30.119534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:30.122521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0f7b5-dbb5-46b1-8796-a63375e5b4d9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:30.125889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 590/1160] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/projects/cdb2fd27b25a4fff8c7ee9c5cc2af8d5 => generated 311 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:30.132784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:30.150010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:30.248794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ada47-d82a-430c-8b93-ad62a8a7583d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:30.288079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:30.288748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:30.289031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:30.289307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:30.290162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5, user.id=9e173bb6886946ffb7064036601974e4) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:30.290510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:30.292007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38ca1a-a0fa-412b-b838-06bae6d8e438 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:30.299099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 571/1161] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:30 2022] GET /identity/v3/role_assignments?user.id=9e173bb6886946ffb7064036601974e4&scope.project.id=cdb2fd27b25a4fff8c7ee9c5cc2af8d5&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:32.424288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4e98-6989-4043-8c84-8ca976b7ac01 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:32.424893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4e98-6989-4043-8c84-8ca976b7ac01 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:32.425152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4e98-6989-4043-8c84-8ca976b7ac01 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:32.427656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 591/1162] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:32.433900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:32.434518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:32.434518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:32.452473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:32.453942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:32.531574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 572/1163] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:32.555020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e4e98-6989-4043-8c84-8ca976b7ac01 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:32.673419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e4e98-6989-4043-8c84-8ca976b7ac01 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:32.688621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:32.714201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:32.714968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:32.715302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:32.715630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:32.792172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:32.793660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:32.907793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 592/1164] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:32.914346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:32.926064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:33.031416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d73c-d38f-44c5-880c-e1e005d3fc08 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:33.071033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:33.071858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:33.072681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:33.072958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:33.081216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:33.082833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0975173e-0d58-428e-8e4b-96a73e5b7e54 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:33.099220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:33.101059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:33.102668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 573/1165] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:26:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:33.107215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:33.123522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:33.218399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e384fe-3b45-4404-a4a7-2dacc7a1332e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:33.257307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:33.257968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:33.258249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:33.258554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:33.259291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:33.260144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:33.261521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590746ce-eb4a-4e21-a19b-19beba526a49 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:33.269652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 593/1166] 10.222.0.77 () {56 vars in 1115 bytes} [Mon Jul 18 16:26:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:35.377252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f67f9-5c98-411d-8377-0ac9212629ef None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:35.377812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f67f9-5c98-411d-8377-0ac9212629ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:35.377812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f67f9-5c98-411d-8377-0ac9212629ef None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:35.379230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 574/1167] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:35.385564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd518271-b990-431d-a71c-4b28d52545da None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:35.386153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd518271-b990-431d-a71c-4b28d52545da None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:35.386290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd518271-b990-431d-a71c-4b28d52545da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:35.405965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bd518271-b990-431d-a71c-4b28d52545da None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:35.407353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bd518271-b990-431d-a71c-4b28d52545da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:35.488382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 594/1168] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:35.512278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f67f9-5c98-411d-8377-0ac9212629ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:35.592986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:35.634484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f67f9-5c98-411d-8377-0ac9212629ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:35.675676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:35.676353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:35.676656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:35.676924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:35.677702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:35.679184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae78f5-eee5-48dd-90c3-2e57739e148f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:35.696462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 575/1169] 10.222.0.77 () {60 vars in 1138 bytes} [Mon Jul 18 16:26:35 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:26:37.871896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d8b6c-2d92-48de-aa8d-b0157d33260a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:37.872488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d8b6c-2d92-48de-aa8d-b0157d33260a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:37.872488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d8b6c-2d92-48de-aa8d-b0157d33260a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:37.873846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 595/1170] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:37.882660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:37.883251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:37.883251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:37.902767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:37.903906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:37.982316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 576/1171] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:38.007358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d8b6c-2d92-48de-aa8d-b0157d33260a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:38.017345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:38.127659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d8b6c-2d92-48de-aa8d-b0157d33260a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:38.171289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:38.172189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:38.172473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:38.172746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] PATH_INFO: `/v3/domains/e96ea7ec969f45219311eab14e46533d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:38.185470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e96ea7ec969f45219311eab14e46533d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:38.186969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb93fb-0e46-4b41-a117-149ba34fdb82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:38.188731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 596/1172] 10.222.0.77 () {56 vars in 1183 bytes} [Mon Jul 18 16:26:38 2022] GET /identity/v3/domains/e96ea7ec969f45219311eab14e46533d => generated 267 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:38.195211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:38.262342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:38.309903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77edbf-4f44-4e52-b917-8f9f772c68e0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:38.348429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:38.349176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:38.349508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:38.349833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:38.350641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:38.352182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2014-9529-45ad-b90c-1df7057c1144 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:38.385077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 577/1173] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:26:38 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:26:40.536884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19706-8552-419b-9ec8-ead9a372f23e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:40.537469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19706-8552-419b-9ec8-ead9a372f23e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:40.537469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19706-8552-419b-9ec8-ead9a372f23e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:40.539008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 597/1174] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:40.546632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:40.547155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:40.547655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:40.575616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:40.577801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:40.691673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 578/1175] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:40.717431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19706-8552-419b-9ec8-ead9a372f23e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:40.791298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:40.835470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19706-8552-419b-9ec8-ead9a372f23e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:40.875460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:40.876506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:40.876900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:40.877311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] PATH_INFO: `/v3/domains/e96ea7ec969f45219311eab14e46533d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:40.878876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e96ea7ec969f45219311eab14e46533d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:40.880435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6904833-c515-41f0-9eb4-1b5cc0176eb7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:40.882164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 598/1176] 10.222.0.77 () {56 vars in 1183 bytes} [Mon Jul 18 16:26:40 2022] GET /identity/v3/domains/e96ea7ec969f45219311eab14e46533d => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:40.888540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:40.938232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:41.006483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724e86-35ba-49da-b0c2-254f1d0a240c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:41.044588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:41.045180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:41.045483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:41.045778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:41.046590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:41.048149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734d509-fa64-4154-8f4e-f2333c8fb891 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:41.070916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 579/1177] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:26:40 2022] POST /identity/v3/users => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:43.315286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534618-d0e5-4108-8001-3dd4513352a1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:43.316102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534618-d0e5-4108-8001-3dd4513352a1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:43.316256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534618-d0e5-4108-8001-3dd4513352a1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:43.317850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 599/1178] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:43.324890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:43.325518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:43.325620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:43.345708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:43.347123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:43.432314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 580/1179] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:43.456719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24534618-d0e5-4108-8001-3dd4513352a1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:43.574310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24534618-d0e5-4108-8001-3dd4513352a1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:43.613425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:43.614226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:43.614580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:43.614857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:43.617988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:43.629754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:43.637876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:43.640017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:26:43.641803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 600/1180] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:43.646231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:43.693226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:43.782221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fc747-e273-42e0-a26b-e34fdd6dfe48 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:43.826468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:43.827141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:43.827676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:43.827949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:43.828829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:43.829228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:43.830703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:43.841210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 581/1181] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:43.850670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:43.963701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016c48-4cd4-436f-99cf-d9403f693c95 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:44.006069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:44.007033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:44.007310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:44.007580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] PATH_INFO: `/v3/users/ce7f1a44fbc44dcc835dd467206e2a03` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:44.024716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] RBAC: Authorizing `identity:get_user(user_id=ce7f1a44fbc44dcc835dd467206e2a03)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:44.026203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:44.028057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 601/1182] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:43 2022] GET /identity/v3/users/ce7f1a44fbc44dcc835dd467206e2a03 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:44.035908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:44.139337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:44.153073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf7bd-a101-4828-bd5d-c44cbb6e11c4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:44.194767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:44.195511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:44.195794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:44.196199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:44.275062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:44.276549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:44.380294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:44.395235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 582/1183] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:44.401972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:44.463799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:44.517765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34393f6-cbf4-48ad-923b-877f90aafb40 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:44.558145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:44.559065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:44.559367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:44.559630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] PATH_INFO: `/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:44.561007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] RBAC: Authorizing `identity:get_project(project_id=e06264922ebb41a58be0fa6e5b1d4f0d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:44.562607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380e249-bc46-4fb5-949d-f79d954cd78f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:44.564412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 602/1184] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:44 2022] GET /identity/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:44.575067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:44.697894 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a94a-85ab-4663-9a06-635663e70447 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:44.738563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:44.739222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:44.739502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:44.739773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:44.740613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=e06264922ebb41a58be0fa6e5b1d4f0d, user.id=ce7f1a44fbc44dcc835dd467206e2a03) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:44.740922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:44.742381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6abcb6-48d0-44e5-a667-1daaa3e1a82b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:44.749143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 583/1185] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:44 2022] GET /identity/v3/role_assignments?user.id=ce7f1a44fbc44dcc835dd467206e2a03&scope.project.id=e06264922ebb41a58be0fa6e5b1d4f0d&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:46.885920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a33e-5559-4096-b3e7-7e59888470f1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:46.886520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a33e-5559-4096-b3e7-7e59888470f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:46.886520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a33e-5559-4096-b3e7-7e59888470f1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:46.887916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 603/1186] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:26:46.893955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:46.894531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:46.894531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:46.912748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:46.913886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:46.969454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:46.991774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 584/1187] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:47.016629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255a33e-5559-4096-b3e7-7e59888470f1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:47.200981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255a33e-5559-4096-b3e7-7e59888470f1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:47.239039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:47.257279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:47.258090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:47.258378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:47.258683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:47.270563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:47.278505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:47.280464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:26:47.282191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 604/1188] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:47.286128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:47.468973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35049ff-0ce1-4f2a-a2b0-c68bf2dbc841 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:47.534027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:47.535217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:47.535716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:47.536216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:47.537636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:47.538186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:47.541512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:47.558852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 585/1189] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:47.566444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:47.696408 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bafedb-a85c-4c0b-8742-c22a65d8f07f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:47.735711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:47.736569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:47.736925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:47.737253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] PATH_INFO: `/v3/users/ce7f1a44fbc44dcc835dd467206e2a03` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:47.738690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] RBAC: Authorizing `identity:get_user(user_id=ce7f1a44fbc44dcc835dd467206e2a03)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:47.740259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:47.742054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 605/1190] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:47 2022] GET /identity/v3/users/ce7f1a44fbc44dcc835dd467206e2a03 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:47.749189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:47.802129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:47.866045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e460acb-2447-4e6b-8f09-0a990ce4ac59 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:47.908459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:47.909519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:47.909707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:47.909989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] PATH_INFO: `/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:47.911401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] RBAC: Authorizing `identity:get_project(project_id=e06264922ebb41a58be0fa6e5b1d4f0d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:47.912907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274971d-d159-49cd-829f-3893bc153196 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:47.914847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 586/1191] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:47 2022] GET /identity/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:47.921597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:48.012655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:48.106362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16952fa-2e3e-4e05-9152-1334b128e31b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:48.170850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:48.171944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:48.172229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:48.172541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] PATH_INFO: `/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d/users/ce7f1a44fbc44dcc835dd467206e2a03/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:48.175000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] RBAC: Authorizing `identity:create_grant(project_id=e06264922ebb41a58be0fa6e5b1d4f0d, user_id=ce7f1a44fbc44dcc835dd467206e2a03, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:48.177201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec669cd4-03cb-470f-be04-8831be95f574 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:48.194470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 606/1192] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:26:47 2022] PUT /identity/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d/users/ce7f1a44fbc44dcc835dd467206e2a03/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:26:50.320267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf201df-3ac9-434e-b2b8-4be260facadf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:50.320895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf201df-3ac9-434e-b2b8-4be260facadf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:50.320895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf201df-3ac9-434e-b2b8-4be260facadf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:50.323050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 587/1193] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:50.329855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:50.330445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:50.330565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:50.349424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:50.350697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:50.427686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 607/1194] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:50.472860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf201df-3ac9-434e-b2b8-4be260facadf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:50.606149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf201df-3ac9-434e-b2b8-4be260facadf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:50.622618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:50.646589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5a751-a608-492a-821f-51d53865954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:50.647429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5a751-a608-492a-821f-51d53865954b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:50.647704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5a751-a608-492a-821f-51d53865954b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:50.647972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5a751-a608-492a-821f-51d53865954b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:50.659736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5a751-a608-492a-821f-51d53865954b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98f5a751-a608-492a-821f-51d53865954b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:50.667739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:50.669485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:26:50.671259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 588/1195] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:50.677416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:50.725203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:50.815883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae92553-03a2-48c1-a493-381263b9b6df None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:50.873688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:50.874376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:50.874763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:50.875114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:50.876094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:50.876484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:50.878203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:50.890300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 608/1196] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:50.899905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5a751-a608-492a-821f-51d53865954b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:51.043252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5a751-a608-492a-821f-51d53865954b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:51.082798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:51.083698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:51.083982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:51.084255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] PATH_INFO: `/v3/users/ce7f1a44fbc44dcc835dd467206e2a03` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:51.085591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] RBAC: Authorizing `identity:get_user(user_id=ce7f1a44fbc44dcc835dd467206e2a03)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:51.087156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:51.088922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 589/1197] 10.222.0.77 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:50 2022] GET /identity/v3/users/ce7f1a44fbc44dcc835dd467206e2a03 => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:51.095346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:51.243949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:51.248063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88ace-b1f8-44f8-b34b-84246c0eaa89 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:51.291349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c623-0670-4e08-985b-664240103415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:51.292028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c623-0670-4e08-985b-664240103415 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:51.292315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c623-0670-4e08-985b-664240103415 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:51.292592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c623-0670-4e08-985b-664240103415 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:51.373473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543c623-0670-4e08-985b-664240103415 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:51.375077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543c623-0670-4e08-985b-664240103415 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:51.504280 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:51.510459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 609/1198] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:26:51.517225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:51.569022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:51.640178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ae-19ef-4e23-b65e-2a3bc2adeca8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:51.681393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:51.684102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:51.684407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:51.684679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] PATH_INFO: `/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:51.686094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] RBAC: Authorizing `identity:get_project(project_id=e06264922ebb41a58be0fa6e5b1d4f0d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:51.687602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483661b0-72ad-4f2a-9921-2930fd389133 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:51.689397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 590/1199] 10.222.0.77 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:51 2022] GET /identity/v3/projects/e06264922ebb41a58be0fa6e5b1d4f0d => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:51.696020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c623-0670-4e08-985b-664240103415 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:51.813265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c623-0670-4e08-985b-664240103415 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:51.858556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:51.859225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:51.859507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:51.859782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:51.860643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=e06264922ebb41a58be0fa6e5b1d4f0d, user.id=ce7f1a44fbc44dcc835dd467206e2a03) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:51.860954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:51.862458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c184af-eba8-4f6b-82fe-780902d90710 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:51.870072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 610/1200] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:51 2022] GET /identity/v3/role_assignments?user.id=ce7f1a44fbc44dcc835dd467206e2a03&scope.project.id=e06264922ebb41a58be0fa6e5b1d4f0d&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:01.676933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=46 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:01.866231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=68 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:54.431454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf0bb0-b975-43fd-8e56-9165e07274ef None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.432365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf0bb0-b975-43fd-8e56-9165e07274ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.432365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf0bb0-b975-43fd-8e56-9165e07274ef None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.433827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 591/1201] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:27:54.445163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.445996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.445996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.480674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:27:54.483612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:27:54.658473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 611/1202] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:27:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:27:54.703646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf0bb0-b975-43fd-8e56-9165e07274ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.925051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf0bb0-b975-43fd-8e56-9165e07274ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.982331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.983976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.984614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.985130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.010500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.013993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:55.047281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:55.048409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:55.049597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:55.049597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:55.049597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:55.049597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:27:55.049597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 592/1203] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:55.056128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.320593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c17cde-b6c4-4973-b1a4-f174af855473 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.367056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.379482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.380655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.381704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.382198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.383669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:55.384169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.387076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3c69e-3542-4556-ae23-997a8a703c6f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.417817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 612/1204] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:55.426988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.534930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.640410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaafe1-f366-415e-ad21-1b693aa69285 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.702326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.703474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.703978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.704471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.705485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.707383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8f2c-94bb-4c28-ba7a-2f5404429e04 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.747453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 593/1205] 10.222.0.77 () {60 vars in 1132 bytes} [Mon Jul 18 16:27:55 2022] POST /identity/v3/users => generated 251 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:27:58.180523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd7d8-63a4-4fd5-9f8c-7b1315a15e43 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.181347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd7d8-63a4-4fd5-9f8c-7b1315a15e43 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.181347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd7d8-63a4-4fd5-9f8c-7b1315a15e43 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.182479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 613/1206] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:27:58.188681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.191128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.191128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.213747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:27:58.215873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:27:58.329965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 594/1207] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:27:58.358960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fd7d8-63a4-4fd5-9f8c-7b1315a15e43 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.510690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fd7d8-63a4-4fd5-9f8c-7b1315a15e43 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.555091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.556451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.556952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.557436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.572278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:27:58.582694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:58.583803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:27:58.584743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 614/1208] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:58.593142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.654587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:58.738737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29d9f6-49d7-4ba5-a046-79f48434071f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.798715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.799424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.799834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.800127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.801209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:58.801527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:58.803896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:58.820295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 595/1209] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:27:58.822499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.898293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:58.977722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07994d3-6d30-4c8c-8b86-7af745bd91ee None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.031110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.032057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.032337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.032648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.065910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.068269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:59.108493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:59.110722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:59.112766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:59.112766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:59.112766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:59.112766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:27:59.112766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 615/1210] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:27:59.128106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.303689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.311547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8495-1222-4084-a19f-9da58d847b4c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.388484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.389579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.390074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.390747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.392890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:59.393043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.395745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.421074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 596/1211] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:59.435287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.617653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badbfc1-2b7f-42ec-bfbe-89dd712ce812 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.620008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.669327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.670365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.670930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.671357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.683427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.686184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:27:59.711707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:59.713039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:27:59.714060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 616/1212] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:59.724307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.897715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2c9e-8045-4928-9717-6e7e2ae67f3c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.964114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.965004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.965415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.965828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.966985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:59.967390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.969286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.996585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 597/1213] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:00.004363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:00.190941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed019b-47b2-4b1b-add6-678179f51dc6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:00.262216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:00.263060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:00.263422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:00.263882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:00.323276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:00.396223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:00.398665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:00.585613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 617/1214] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:00.591904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:00.632571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:00.797846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7daec-a289-4d44-ac07-c9799fb70673 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:00.864512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:00.866762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:00.867320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:00.867706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:00.893803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:00.895595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:00.923150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:00.925231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:00.927417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:00.927417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:00.927417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:00.927417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:00.931630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 598/1215] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:00.932494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:01.054267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:01.108985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca9b6-e530-4deb-bb25-534bb952179b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:01.160565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:01.161649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:01.161955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:01.162250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:01.163718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:01.164035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:01.165796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:01.195613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 618/1216] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:01.197613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:01.312229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:01.359597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee93b6b-803b-4fb6-b8ab-5f46790ab178 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:01.413607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:01.415071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:01.415577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:01.416049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:01.429955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:01.433184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:01.456708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:01.458579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:01.461756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:01.463067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 599/1217] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:01.468948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:01.671522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c500a-5475-4da4-9fd0-4f836708f172 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:01.719465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:01.741200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:01.742537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:01.743052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:01.743526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:01.745242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:01.745799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:01.748859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81aade-53af-45da-b7c5-7414866f0599 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:01.771359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 619/1218] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:01.777748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:01.956856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c917f-aea5-43fb-95e5-30c90f44e298 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:02.007900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:02.022576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:02.023901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.024481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.025011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.027075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=52152797f2784be389bfae5dd38ef5d8) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:02.027748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:02.031193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960923-90a5-41e6-82fd-3991548a96c4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:02.054445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 600/1219] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:01 2022] GET /identity/v3/role_assignments?user.id=52152797f2784be389bfae5dd38ef5d8&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:04.423337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854e93-117d-420a-b130-ffdca887aac8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.424364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854e93-117d-420a-b130-ffdca887aac8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.424364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854e93-117d-420a-b130-ffdca887aac8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.426343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 620/1220] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:04.433579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.434258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.434258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.456622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:04.458780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:04.562479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 601/1221] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:04.609345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91854e93-117d-420a-b130-ffdca887aac8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:04.807711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91854e93-117d-420a-b130-ffdca887aac8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:04.866524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:04.867664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.868147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.868562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.884456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:04.897337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:04.898661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:04.899544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:04.905739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 621/1222] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:28:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:04.907395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:05.098120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eb235-d48f-4938-b500-91f3cfca5592 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:05.142120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:05.162574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:05.164113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:05.164933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:05.165303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:05.166804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:05.167353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:05.170010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:05.193681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 602/1223] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:28:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:05.195252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:05.256238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:05.327450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416fa1-84d5-49cb-ab27-65700a1c4960 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:05.374474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:05.375376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:05.375757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:05.376157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:05.391694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:05.393388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:05.415319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:05.417253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:05.419541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:05.419541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:05.419541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:05.419541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:05.419541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 622/1224] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:05.424455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:05.578565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cba682-284d-40de-940a-e19ef7950ac1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:05.637379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:05.638399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:05.638926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:05.639432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:05.641102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:05.641648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:05.644376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:05.669603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 603/1225] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:05.679215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:05.845777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:05.854458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543791d-5fed-4a0c-a151-941aa4c3289a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:05.892638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:05.893642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:05.894141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:05.894514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:05.907840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:05.910712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:05.937639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:05.940002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:05.942021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 623/1226] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:05.949868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:06.023541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:06.114744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6936-d091-4fa0-8886-eb6afdfd5fa6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:06.166711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:06.167576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.167986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.168376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.169510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:06.169938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:06.171612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:06.205645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 604/1227] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:06.215389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:06.422311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f04c26-8e22-449b-837e-d3387d471db3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:06.482285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:06.483286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.483731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.484112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.501461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:06.503934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:06.549851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:06.551828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:06.553942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:06.553942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:06.553942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:06.553942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:06.553942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 624/1228] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:06.561491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:06.709203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e35c73-8c2d-49fb-aacb-768e7acf51bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:06.712167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:06.756643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:06.757575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.757880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.758162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.759182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:06.759547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:06.760990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:06.781347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 605/1229] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:06.788473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:06.807765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:06.951381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3bbf1-a445-49e7-bd85-420ab85e6c66 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:06.993734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:06.994503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.994793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.995065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.009266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.012372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:07.035287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:07.036453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:07.037584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 625/1230] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:07.047258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.194204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457fa6-7644-4b1b-ac48-cc09b2a0c8e2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.236822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.237565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.237850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.238153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.239016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:07.239325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.241440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a8ef0-f081-4f99-9507-060e1a0a87b6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.256872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 606/1231] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:07.263986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.379351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:07.448116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54933c3e-986b-4f24-9f2f-7a0e5bbe7702 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.513647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.514743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.515253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.515732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/52152797f2784be389bfae5dd38ef5d8/roles/bd21cadba0de49c79bbca196eb4b5d32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.542574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=52152797f2784be389bfae5dd38ef5d8, role_id=bd21cadba0de49c79bbca196eb4b5d32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.545293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0429d0-2d67-470b-80fd-8d796a38b9c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.564953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 626/1232] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:28:07 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/52152797f2784be389bfae5dd38ef5d8/roles/bd21cadba0de49c79bbca196eb4b5d32 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:28:09.916235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42247518-ba26-4767-bab1-1069a97ef45e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:09.916946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42247518-ba26-4767-bab1-1069a97ef45e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:09.916946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42247518-ba26-4767-bab1-1069a97ef45e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:09.918840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 607/1233] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:28:09.925304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:09.925872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:09.925872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:09.944621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:09.945877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:10.042279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 627/1234] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:10.067408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42247518-ba26-4767-bab1-1069a97ef45e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:10.104594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:10.204920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42247518-ba26-4767-bab1-1069a97ef45e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:10.249478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:10.250750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:10.251102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:10.251407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:10.268966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:10.280162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:10.281959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:10.283668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 608/1235] 10.222.0.77 () {56 vars in 1102 bytes} [Mon Jul 18 16:28:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:10.292776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:10.356362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:10.430084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa54a9-5587-4e6d-99d1-6982c8e142c2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:10.480296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:10.481014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:10.481439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:10.481928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:10.483032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:10.483354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:10.485037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:10.496031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 628/1236] 10.222.0.77 () {56 vars in 1103 bytes} [Mon Jul 18 16:28:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:10.502505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:10.642617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:10.673761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df866c9-db8e-45b7-bdf9-730032c640f6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:10.745732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:10.747221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:10.747703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:10.748207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:10.770072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:10.773079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:10.814945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:10.816554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:10.817688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:10.817688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:10.817688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:10.817688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:10.817688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 609/1237] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:10.829213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:10.982113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874f1c-7c69-4cbf-b127-753461d9ffff None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:10.993638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.028202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.028998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.029298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.029613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.030530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:11.030936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.032598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.049999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 629/1238] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:11.057190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.211274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9caf-a8bb-44b3-b1c6-fdbded7d3836 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.272082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.273054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.273413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.273747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.277882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.287388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.289998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:11.315889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:11.318602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:11.320478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 610/1239] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:11.327092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.492759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e6bf5-f08e-4995-954b-35b4cfe155f3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.545285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.546337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.546793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.547204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.548349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:11.548774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.551617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.588961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 630/1240] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:11.598217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.810170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa24a8c-beb5-473f-a7c2-8e1da9a8c2d4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.888409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.889677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.890204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.890725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.982263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.992971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.994822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.141148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 611/1241] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:12.151807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.196381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.302179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4dbd6-1205-4a5e-a4a7-80db0e657445 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.347996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.348835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.349256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.349423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.361111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.362644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:12.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:12.387635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:12.388739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:12.388739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:12.388739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:12.388739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:12.394959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 631/1242] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:12.396793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.525341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.535028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64031b-bf73-483b-be6d-46812e7a418e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.591685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.592586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.593021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.593450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.594310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:12.594747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.596316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.613542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 612/1243] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:12.620946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.753361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.796921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1255c1-5b24-4e61-ade7-d6a3ace5366c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.855417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.856243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.856530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.856833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.865960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.867516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:12.884339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:12.886300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:12.888065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 632/1244] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:12.893853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:13.054162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116128-a5a7-470a-99e9-03d61ea48598 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:13.101404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:13.102095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:13.102382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:13.102696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:13.103531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:13.103847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:13.105379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ed6ae-4276-4d20-b182-681c860513ce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:13.121783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 613/1245] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:13.128819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:13.282483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3792cf-8e0c-4dbc-929c-51969c0fa8c0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:13.337507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:13.338529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:13.338954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:13.339343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:13.340473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] RBAC: Adding query filter params (role.id=bd21cadba0de49c79bbca196eb4b5d32, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=52152797f2784be389bfae5dd38ef5d8) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:13.340887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:13.342801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b3e3-255b-4dd2-8471-c09105934554 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:13.352843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 633/1246] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:13 2022] GET /identity/v3/role_assignments?user.id=52152797f2784be389bfae5dd38ef5d8&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=bd21cadba0de49c79bbca196eb4b5d32 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:15.879963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b751fd-3d6a-47f1-96b7-d7117d013c40 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.881009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b751fd-3d6a-47f1-96b7-d7117d013c40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.881009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b751fd-3d6a-47f1-96b7-d7117d013c40 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.883016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 614/1247] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:15.891786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.892427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.892427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.901082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:15.919293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:15.920976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:16.020149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 634/1248] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:16.050005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b751fd-3d6a-47f1-96b7-d7117d013c40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.075626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.211287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b751fd-3d6a-47f1-96b7-d7117d013c40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.265829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.267294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.267777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.268268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.287276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:16.298983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:16.301173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:16.303267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 615/1249] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:16.309717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.482398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225820b-124d-4f85-bfbc-68bc6d81fa9f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.528815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.535101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.536125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.537042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.537248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.538058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:16.538382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.539978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.552750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 635/1250] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:16.560333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.682242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.717829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c8fb5-5516-4393-b558-be73248a77af None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.773446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.774789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.775282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.775951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.798443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.801316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:16.837008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:16.838696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:16.845973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:16.845973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:16.845973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:16.845973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:16.845973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 616/1251] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:16.847401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.996146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d6634-0c50-4627-ac44-20116849edb9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.048340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.049001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.049302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.049612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.050528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:17.050844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.052316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.069672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 636/1252] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:17.077371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.253756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c6f88-bf9c-4bb9-96b3-207ad9b94424 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.290855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:17.303398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.304598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.305055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.305471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.321625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.324382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:17.354704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:17.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:17.357579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 617/1253] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:17.366230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.465996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:17.536089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb73cb-a0c3-4b20-ab32-669f45f31831 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.603774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.605058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.605621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.606172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.607810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:17.608259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.610766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.649869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 637/1254] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:17.658832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.808289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5d340-94e6-47d1-93e2-2568e5c66a62 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.868770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.869915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.870545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.871112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.979074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:18.001242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:18.003476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:18.152725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 618/1255] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:18.159170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:18.327595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89365302-c4b4-423d-84c6-bff6ae88376c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:18.353396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:18.397684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:18.398921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:18.399322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:18.399729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:18.417812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:18.419765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:18.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:18.450555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:18.452259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:18.452259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:18.452259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:18.452259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:18.452259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 638/1256] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:18.459180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:18.591264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:18.608416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974ad66-5f00-4114-b8cf-0a092fc10696 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:18.652273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:18.652976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:18.653221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:18.653499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:18.654298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:18.654619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:18.656066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:18.674086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 619/1257] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:18.682325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:18.853874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013f79-49d9-4ebb-bff8-13e9d2b8dcbc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:18.916420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:18.917836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:18.918339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:18.918874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:18.932160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:18.934277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:18.952303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:18.954066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:18.955762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 639/1258] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:18.961057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:19.140077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae2cb0-9f9a-4ad8-b581-119d965d8f3f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:19.197423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:19.198571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.198988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.199543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.201201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:19.201765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:19.204226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62a438-29ce-4239-bbdc-73f6ae80f866 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:19.227347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 620/1259] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:19.229798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:19.266557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:19.360710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e315e5-9cb8-490c-86b3-1a867ad3fa2a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:19.409489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:19.410281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.410688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.410923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.411979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=52152797f2784be389bfae5dd38ef5d8) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:19.412311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:19.414155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c66840-7d6b-4ce5-9a37-2a7e96e78431 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:19.422171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 640/1260] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:19 2022] GET /identity/v3/role_assignments?user.id=52152797f2784be389bfae5dd38ef5d8&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:21.783400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef164f-b070-42cf-a4cc-f4b19e4e5703 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:21.784067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef164f-b070-42cf-a4cc-f4b19e4e5703 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:21.784067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef164f-b070-42cf-a4cc-f4b19e4e5703 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:21.785620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 621/1261] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:21.793269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:21.793826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:21.794227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:21.823724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:21.826296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:21.965441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 641/1262] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:28:21.988403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef164f-b070-42cf-a4cc-f4b19e4e5703 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:22.001393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.173995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef164f-b070-42cf-a4cc-f4b19e4e5703 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:22.237116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:22.238822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.239351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.240332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.260318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:22.275271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:22.277135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:22.279111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 622/1263] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:22.285636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:22.308520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.402196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425362c-b3ea-42d7-8c21-c6e47722a0df None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:22.441365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:22.442054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.442333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.442677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.443434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:22.443734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:22.445139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:22.454825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 642/1264] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:22.464503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:22.556070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.597457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d91f5-f290-42fb-a35c-6693696b4f17 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:22.637399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:22.638201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.638506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.638782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.652309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:22.653784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:22.673294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:22.675164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:22.676818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:22.676818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:22.676818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:22.676818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:22.676818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 623/1265] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:22.681172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:22.774124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.799108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f5890-96a7-45bf-a63a-00504bc859ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:22.855159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:22.855991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.856349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.856878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.858166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:22.858654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:22.860343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:22.875191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 643/1266] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:22.881907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.001438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892d1b0-1b41-46dd-b603-e85605c067bc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:23.024226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:23.040756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:23.041576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:23.041851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:23.042122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:23.063037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:23.064533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:23.080804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:23.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:23.084199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 624/1267] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:23.091504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.207428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66791fda-ceae-4d54-9558-adec49534cba None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:23.247534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:23.248218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:23.248501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:23.248776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:23.249632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:23.249942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:23.251430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:23.262084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:23.272153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 644/1268] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:23.278351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.394813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2151c-271d-44ad-a306-48dff9b9668b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:23.433932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:23.434808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:23.435226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:23.435505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:23.449607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:23.451519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:23.471845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:23.473749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:23.475743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:23.475743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:23.475743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:23.475743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:23.475743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 625/1269] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:23.481135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.618476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989673e-8d94-44d1-a046-f34ced559dd3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:23.657181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:23.657857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:23.658141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:23.658443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:23.659200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:23.659509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:23.660928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:23.674360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 645/1270] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:23.680504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.716568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:23.795331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5a7b-93c4-4474-9477-ecdd747abb35 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:23.834094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:23.834933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:23.835215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:23.835488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:23.843204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:23.844688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:23.860257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:23.861981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:23.863347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 626/1271] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:23.868425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:23.962587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:24.013177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f8537-acc0-4ada-b761-ae71c3c3d259 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:24.052929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:24.053517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:24.053796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:24.054068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:24.054920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:24.055230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:24.056721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8fe2-6c8a-4126-af19-2de03cfef27d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:24.071219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 646/1272] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:24.077960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:24.182913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:24.194235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82da4b-588d-44aa-8d42-08dfdc66f4b8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:24.231677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:24.232635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:24.232959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:24.233233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] PATH_INFO: `/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/52152797f2784be389bfae5dd38ef5d8/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:24.235327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] RBAC: Authorizing `identity:create_grant(project_id=8e91ceaccc1f4cc5a51c17a9216967ea, user_id=52152797f2784be389bfae5dd38ef5d8, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:24.236817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c51f4a-3ea0-42d8-a8ea-5c071777d43d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:24.259215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 627/1273] 10.222.0.77 () {58 vars in 1439 bytes} [Mon Jul 18 16:28:24 2022] PUT /identity/v3/projects/8e91ceaccc1f4cc5a51c17a9216967ea/users/52152797f2784be389bfae5dd38ef5d8/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:28:26.404295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852730-0089-41c7-b91a-4cbbdb9e2b9a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:26.404908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852730-0089-41c7-b91a-4cbbdb9e2b9a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:26.404908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852730-0089-41c7-b91a-4cbbdb9e2b9a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:26.406275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 647/1274] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:26.413052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:26.413658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:26.413658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:26.432200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:26.433506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:26.512792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 628/1275] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:28:26.537100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86852730-0089-41c7-b91a-4cbbdb9e2b9a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:26.656215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86852730-0089-41c7-b91a-4cbbdb9e2b9a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:26.676956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:26.696229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:26.697100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:26.697424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:26.697743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:26.711715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:26.720005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:26.721835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:26.724385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 648/1276] 10.222.0.77 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:26.730119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:26.841350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:26.867925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350133e-1a8f-4cf7-833b-f8dc09522e70 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:26.913522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:26.914281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:26.914641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:26.914959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:26.915805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:26.916154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:26.917635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:26.927581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 629/1277] 10.222.0.77 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:26.934034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:27.052486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30149-b103-44d9-b416-ed9ab57fdb26 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:27.094004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:27.094894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:27.095210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:27.095527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:27.108934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:27.110519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:27.130164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:27.131967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:27.133782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:27.133782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:27.133782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:27.133782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:27.133782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 649/1278] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:27.137743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:27.252100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839665-9911-497b-9025-d9d2e52c41cd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:27.290234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:27.291057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:27.291378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:27.291689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:27.292502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:27.292850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:27.294305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:27.307960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 630/1279] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:27.313959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:27.350667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:27.434980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0bea8-142d-46e1-975e-848ed48a5cd9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:27.473420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:27.474881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:27.475214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:27.475531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:27.484700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:27.486218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:27.502296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:27.504133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:27.506932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 650/1280] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:27.512251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:27.516650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:27.626853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619b262-7f95-45a2-ac0f-5a7ba93efeac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:27.667455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:27.668148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:27.668432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:27.668708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:27.669564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:27.669875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:27.671365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:27.692138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 631/1281] 10.222.0.77 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:27.698351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:27.816192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:27.830012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119678c-2f58-4f55-8869-a122e4e34fa9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:27.869758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:27.870521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:27.870850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:27.871186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:27.950390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:27.952010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:28.054625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:28.068427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 651/1282] 10.222.0.77 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:28.074631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:28.146516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:28.193559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d6baf-0a48-43e5-8e8d-155dc75ee9e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:28.234349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:28.235209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.235876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.236150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.248722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:28.250227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:28.269496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:28.270782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:28.271958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:28.271958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:28.271958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:28.271958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:28.271958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 632/1283] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:28.276756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:28.431596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49282c5e-8ffc-4f20-9335-5df040dd2c6c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:28.469473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:28.470206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.470429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.470704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.471484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:28.471785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:28.473167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:28.486009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 652/1284] 10.222.0.77 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:28.492320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:28.608817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728833-b358-4c93-80e4-f83f53961db9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:28.626009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:28.650702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:28.651523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.651804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.652075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.660057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:28.661564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:28.677284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:28.679408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jul 18 16:28:28.681907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 633/1285] 10.222.0.77 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:28.687561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:28.759508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:28.821443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091267ee-32a6-4cf8-8e5f-bdfcaa087918 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:28.860626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:28.861302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.861582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.861864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.862708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:28.863015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:28.865033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d147-4434-4b93-87cc-5756af53f668 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:28.878988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 653/1286] 10.222.0.77 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:28.885996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:29.002755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fffa2f-d764-4e64-b568-4926175ef422 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:29.043113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76579abb-993b-43af-9475-e062751b8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:29.043768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76579abb-993b-43af-9475-e062751b8231 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:29.044048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76579abb-993b-43af-9475-e062751b8231 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:29.044321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76579abb-993b-43af-9475-e062751b8231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:29.045175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76579abb-993b-43af-9475-e062751b8231 None admin] RBAC: Adding query filter params (role.id=54d519e3caa64c32a9930f925d38c156, scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea, user.id=52152797f2784be389bfae5dd38ef5d8) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:29.045487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76579abb-993b-43af-9475-e062751b8231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:29.047000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76579abb-993b-43af-9475-e062751b8231 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:29.054279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 634/1287] 10.222.0.77 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/role_assignments?user.id=52152797f2784be389bfae5dd38ef5d8&scope.project.id=8e91ceaccc1f4cc5a51c17a9216967ea&role.id=54d519e3caa64c32a9930f925d38c156 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:31.155368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462c1ea-78a1-4702-8681-8aa7edf1c352 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:31.155884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462c1ea-78a1-4702-8681-8aa7edf1c352 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:31.156112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462c1ea-78a1-4702-8681-8aa7edf1c352 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:31.157802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 654/1288] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:31.163818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:31.164844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:31.164985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:31.183267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:31.184486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:31.220508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:31.264261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 635/1289] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:31.287126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462c1ea-78a1-4702-8681-8aa7edf1c352 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:31.400999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462c1ea-78a1-4702-8681-8aa7edf1c352 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:31.413219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:31.439367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:31.440199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:31.440482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:31.440754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:31.441431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:31.443418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:31.457131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:31.459737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:31.461545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 655/1290] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:31.465543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:31.585657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11eadb7-cb1e-4f21-a10b-2251f70598fc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:31.626802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:31.627482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:31.627762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:31.628042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:31.628768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:31.629075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:31.631104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01b5b8-91ed-4fff-bb02-1a59b3f63701 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:31.637191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 636/1291] 10.222.0.77 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:31.646058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:31.777504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4c55d-90dc-4211-b1d5-01f45529a4ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:31.820603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:31.821292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:31.821582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:31.821862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:31.822621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:31.822934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:31.824349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022eedb1-a9f7-4941-a00f-de78dcb7b45f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:31.830106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 656/1292] 10.222.0.77 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:33.999734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c51fe1-495f-4818-a07d-9c12bbbb916a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:34.000331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c51fe1-495f-4818-a07d-9c12bbbb916a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:34.000331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c51fe1-495f-4818-a07d-9c12bbbb916a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:34.001746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 637/1293] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:34.007915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f650-7e7c-4186-8c35-966f9e32c574 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:34.008454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f650-7e7c-4186-8c35-966f9e32c574 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:34.008454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f650-7e7c-4186-8c35-966f9e32c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:34.036449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8e28f650-7e7c-4186-8c35-966f9e32c574 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:34.037756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8e28f650-7e7c-4186-8c35-966f9e32c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:34.071702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:34.117511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 657/1294] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:28:34.140391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c51fe1-495f-4818-a07d-9c12bbbb916a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:34.175960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:34.261555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c51fe1-495f-4818-a07d-9c12bbbb916a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:34.298682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:34.299351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:34.299635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:34.299913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:34.300665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:34.302099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30000deb-fd34-41f0-b3e3-2d816d0f116a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:34.316892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 638/1295] 10.222.0.77 () {60 vars in 1141 bytes} [Mon Jul 18 16:28:34 2022] POST /identity/v3/services => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:28:36.429722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78964ad-7af3-4641-9844-cc6ce405cf54 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:36.430450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78964ad-7af3-4641-9844-cc6ce405cf54 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:36.430450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78964ad-7af3-4641-9844-cc6ce405cf54 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:36.431905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 658/1296] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:28:36.438959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:36.439502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:36.439953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:36.467556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:36.469507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:36.603087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 639/1297] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:36.627213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78964ad-7af3-4641-9844-cc6ce405cf54 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:36.745793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78964ad-7af3-4641-9844-cc6ce405cf54 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:36.805003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:36.806390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:36.806926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:36.807388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:36.808641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:36.811364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:36.835366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:36.836584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:36.837474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:36.838258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 659/1298] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:36.845418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:36.872111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:36.970114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a45e1-6a09-4bed-90ed-0a0e2ae2a307 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:37.007572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:37.008161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:37.008442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:37.008721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:37.009437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:37.009744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:37.011184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:37.017352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 640/1299] 10.222.0.77 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:37.027474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:37.065253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:37.187553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e386d-59e9-49f8-a444-16763d355e19 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:37.250355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:37.251520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:37.252034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:37.252538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:37.254090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] RBAC: Adding query filter params (interface=public, service_id=ce9b4e2567564d678bbc2a0e7a718115, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:37.255207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:37.257915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c383262-b005-4816-abe5-942f495c0877 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:37.268032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 660/1300] 10.222.0.77 () {56 vars in 1251 bytes} [Mon Jul 18 16:28:37 2022] GET /identity/v3/endpoints?service_id=ce9b4e2567564d678bbc2a0e7a718115&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:37.274665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:37.372476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:37.394905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644d8db-97b2-485f-9311-4e5b87166bc8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:37.435425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:37.436094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:37.436371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:37.436638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:37.437328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:37.438771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd260b0-8290-4c7f-b926-5bfa3eba6609 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:37.446546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 641/1301] 10.222.0.77 () {56 vars in 1087 bytes} [Mon Jul 18 16:28:37 2022] GET /identity/v3/services => generated 2191 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:28:39.530283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8582-beb3-4dc0-8798-123a35f91004 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:39.530869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8582-beb3-4dc0-8798-123a35f91004 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:39.530869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8582-beb3-4dc0-8798-123a35f91004 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:39.532262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 661/1302] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:28:39.538898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:39.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:39.539484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:39.557375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:39.558615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:39.648650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 642/1303] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:39.678663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8582-beb3-4dc0-8798-123a35f91004 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:39.834668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:39.842449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8582-beb3-4dc0-8798-123a35f91004 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:39.881462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:39.882304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:39.882653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:39.882938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:39.883621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:39.885110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:39.899242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:39.900518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jul 18 16:28:39.901756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 662/1304] 10.222.0.77 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:39.906928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:39.998053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:40.024161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364a269-ac65-4567-9bc5-db185961a6d7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:40.063447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:40.064129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:40.064410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:40.064683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:40.065411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:40.065723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:40.067187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d982f-8408-471e-bb4d-cd586b865278 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:40.073820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 643/1305] 10.222.0.77 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:40.080326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:40.193684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a916c-a95f-4d9c-bc9a-ac52c93826fc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:40.234486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:40.235182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:40.235445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:40.235722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:40.236492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:40.237884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21909cb-d461-4cc0-8521-b08469601f32 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:40.258772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 663/1306] 10.222.0.77 () {60 vars in 1144 bytes} [Mon Jul 18 16:28:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:28:50.069875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=30 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:50.253308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.758487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9c7b-0876-45dd-957b-65f1336ce691 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.759416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9c7b-0876-45dd-957b-65f1336ce691 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.759860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e9c7b-0876-45dd-957b-65f1336ce691 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.766071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 644/1307] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:29:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:52.768662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b8faa-f0c7-46cd-9b9c-58d7e9898564 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.769196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b8faa-f0c7-46cd-9b9c-58d7e9898564 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.769609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b8faa-f0c7-46cd-9b9c-58d7e9898564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.884530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-031b8faa-f0c7-46cd-9b9c-58d7e9898564 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:52.886502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-031b8faa-f0c7-46cd-9b9c-58d7e9898564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.025782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 664/1308] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:29:53.080701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.081281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.081740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.083993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 645/1309] 10.222.0.77 () {54 vars in 965 bytes} [Mon Jul 18 16:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:53.090991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b6c02-f5f4-4bab-9712-85f0fb4f4516 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.091578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b6c02-f5f4-4bab-9712-85f0fb4f4516 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.092013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b6c02-f5f4-4bab-9712-85f0fb4f4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.194659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3a5b6c02-f5f4-4bab-9712-85f0fb4f4516 None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:53.195980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a5b6c02-f5f4-4bab-9712-85f0fb4f4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.223925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:53.304881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 665/1310] 10.222.0.77 () {58 vars in 1062 bytes} [Mon Jul 18 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:29:53.314313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:53.316530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.453338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857f333-160c-4d9a-b596-4d1d826eacef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:53.556854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:53.557641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.557996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.558277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.560193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.643481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:53.768134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:53.771505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9b6f-35ca-4a2f-a869-e544a1c1dc88 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:53.814398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 646/1311] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:29:56.585355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.585935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.585935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.590286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 666/1312] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:56.593479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469ee47-b2ba-4700-9653-f5143e411a77 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.594011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469ee47-b2ba-4700-9653-f5143e411a77 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.594011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469ee47-b2ba-4700-9653-f5143e411a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.674105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7469ee47-b2ba-4700-9653-f5143e411a77 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:56.675396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7469ee47-b2ba-4700-9653-f5143e411a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:56.793087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 647/1313] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:29:56.840184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:56.842530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:57.043746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8868f03-795a-4ebc-a2a0-6b57cccad616 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.130717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.143962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.145032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.145532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.146019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.149271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:57.301453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.302865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92184b9f-a349-459d-b16c-367217785cf4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.327486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 667/1314] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:30:00.275534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271becd-aae1-4bc8-b1d4-385443930294 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.276079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271becd-aae1-4bc8-b1d4-385443930294 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.276079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271becd-aae1-4bc8-b1d4-385443930294 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.277607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 648/1315] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:00.284085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94e276-6930-4f35-a64c-3567712a590a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.284623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94e276-6930-4f35-a64c-3567712a590a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.284795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94e276-6930-4f35-a64c-3567712a590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.362727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6e94e276-6930-4f35-a64c-3567712a590a None None] MFA Rules not processed for user `623c485832d44648b870a433f97830f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:00.363983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6e94e276-6930-4f35-a64c-3567712a590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:00.428507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.485892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 668/1316] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:00.544042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.544271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.544599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.546149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 649/1317] 10.222.0.77 () {54 vars in 965 bytes} [Mon Jul 18 16:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:00.553674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315d92-880d-4434-be3c-9c3c4335b4b3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.553971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315d92-880d-4434-be3c-9c3c4335b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.554258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b315d92-880d-4434-be3c-9c3c4335b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.645584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8b315d92-880d-4434-be3c-9c3c4335b4b3 None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:00.647120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b315d92-880d-4434-be3c-9c3c4335b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:00.739607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 669/1318] 10.222.0.77 () {58 vars in 1062 bytes} [Mon Jul 18 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:00.749511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.751109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:00.762036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.903231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b89fc4-92d5-40b7-8c31-0aae23233cbd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.991510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.992250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.992580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.992903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.994795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:01.071099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:01.127251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:01.128747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef577f-e636-4fa2-aa0a-378e214bd889 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:01.150567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 650/1319] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:04.336734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351db8-7472-433b-988c-b576306fb371 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.337341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351db8-7472-433b-988c-b576306fb371 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.337341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351db8-7472-433b-988c-b576306fb371 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.338993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 670/1320] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:04.345116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463992a-a29b-41fd-91ba-d07ecb62cfd3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.345733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463992a-a29b-41fd-91ba-d07ecb62cfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.345733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463992a-a29b-41fd-91ba-d07ecb62cfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.423630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2463992a-a29b-41fd-91ba-d07ecb62cfd3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:04.424892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2463992a-a29b-41fd-91ba-d07ecb62cfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:04.545644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 651/1321] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:04.587731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351db8-7472-433b-988c-b576306fb371 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.589287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-62351db8-7472-433b-988c-b576306fb371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:04.650751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.712091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351db8-7472-433b-988c-b576306fb371 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.793290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.794079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.794379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.794685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.796530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:04.908526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.924726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.926186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159629b-3589-443b-9b0b-de97a9d7b441 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.946758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 671/1322] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:07.922547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.923348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.923348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.924799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 652/1323] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:07.931777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad02f-7404-4a04-b44b-51dfa472722b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.932614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad02f-7404-4a04-b44b-51dfa472722b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.932914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad02f-7404-4a04-b44b-51dfa472722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.035505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8f8ad02f-7404-4a04-b44b-51dfa472722b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:08.037051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8f8ad02f-7404-4a04-b44b-51dfa472722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:08.167323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 672/1324] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:08.209061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.211028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:08.235345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:08.395416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0648aa-c67e-49a8-b7b8-b1ee836c6c00 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.506535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.507121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.507403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.507678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.509503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:08.598758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:08.665611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.667128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e4a5f-2cd2-48a4-b903-98961585e373 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.692080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 653/1325] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:30:11.402116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.402952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.402952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.404138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 673/1326] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:11.411935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e8fd4-360a-44db-846d-e117f78b8e3b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.412517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e8fd4-360a-44db-846d-e117f78b8e3b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.412925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e8fd4-360a-44db-846d-e117f78b8e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.500303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b12e8fd4-360a-44db-846d-e117f78b8e3b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:11.501571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b12e8fd4-360a-44db-846d-e117f78b8e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:11.626037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 654/1327] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:11.667191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.668887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:11.787739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.876898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8d955-e0f9-4344-81a1-80ddc138caca None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.999329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.999994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.000820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.001090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.002900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:12.144195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.151862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.154668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbd9f6-2a87-4ae5-85fe-c97e12a498e5 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.186688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 674/1328] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:14.627496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.628341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.628530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.630122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 655/1329] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:14.636745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa00076-f5e4-4596-9747-142d7a7cc1cb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.637583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa00076-f5e4-4596-9747-142d7a7cc1cb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.637583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa00076-f5e4-4596-9747-142d7a7cc1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.733014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-afa00076-f5e4-4596-9747-142d7a7cc1cb None None] MFA Rules not processed for user `623c485832d44648b870a433f97830f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:14.734237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-afa00076-f5e4-4596-9747-142d7a7cc1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:14.892226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 675/1330] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:30:14.931578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.933280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:14.970097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.101008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de31e90-0482-4ac9-8065-02f26141f9c1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.246010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.247174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.247683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.248200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.252209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:15.379349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.452202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.455359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a057e-c662-45bd-b94d-40f25f555ff2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.494515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 656/1331] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:30:18.147469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:18.148164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:18.148442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:18.151204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 676/1332] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:18.157633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9b0c-ffde-4e2b-bb63-3ed7d25e81eb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:18.158373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9b0c-ffde-4e2b-bb63-3ed7d25e81eb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:18.158373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9b0c-ffde-4e2b-bb63-3ed7d25e81eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:18.273963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c39e9b0c-ffde-4e2b-bb63-3ed7d25e81eb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:18.275493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c39e9b0c-ffde-4e2b-bb63-3ed7d25e81eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:18.431503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 657/1333] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:18.476772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:18.478497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:18.559629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:18.605476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c0846-3159-4c0b-9b64-d266ddc6b136 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:18.700745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:18.701455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:18.701813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:18.702162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:18.705165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:18.847520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:18.849302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c646bd-4082-4806-99c6-e6fb0c3a16bf service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:18.862785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:18.873479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 677/1334] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:21.096028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.096671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.096671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.098011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 658/1335] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:21.104464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106bba-0729-493e-bd22-6f0c39da243a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.105017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106bba-0729-493e-bd22-6f0c39da243a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.105189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106bba-0729-493e-bd22-6f0c39da243a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.186643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-01106bba-0729-493e-bd22-6f0c39da243a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:21.188493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-01106bba-0729-493e-bd22-6f0c39da243a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:21.299996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 678/1336] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:21.343501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.345024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:21.394737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.475638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e346b7-0e46-4850-83a6-1d79006ea6c3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.561764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.562487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.562772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.563047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.564790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:21.667556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.699705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.701184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf573b-cd4e-4845-a267-23fca11e5746 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.722482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 659/1337] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:24.509590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.510362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.510651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.513374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 679/1338] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:24.522471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357750e5-e9ab-4ab7-97dc-b50f09dc0f83 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.523327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357750e5-e9ab-4ab7-97dc-b50f09dc0f83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.523327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357750e5-e9ab-4ab7-97dc-b50f09dc0f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.616660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-357750e5-e9ab-4ab7-97dc-b50f09dc0f83 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:24.617963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-357750e5-e9ab-4ab7-97dc-b50f09dc0f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:24.772921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 660/1339] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:24.822364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.824824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:24.929223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:24.964320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb0732-7e1f-4e96-8f53-b387ca39a5e4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.053205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.053847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.054174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.054560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.056336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:25.209147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.210819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b6e8-517b-4f6e-9bb8-b65c99e2186b service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.235618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 680/1340] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:29.815377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.815940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.816530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.820227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 661/1341] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:29.826783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdddee2-7468-494a-9bf9-cb8bae228e13 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.827680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdddee2-7468-494a-9bf9-cb8bae228e13 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.827680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdddee2-7468-494a-9bf9-cb8bae228e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.831880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.923944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cbdddee2-7468-494a-9bf9-cb8bae228e13 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:29.925327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cbdddee2-7468-494a-9bf9-cb8bae228e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:30.048915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 681/1342] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:30.089425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.091024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:30.149111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.215592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622bf77-fe5e-4b8e-a841-19eaf80652d3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:30.306026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.306776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.307058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.307331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.309175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:30.423963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.444166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:30.445640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b10396-6915-41cc-b8cc-6859175fef55 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:30.472446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 662/1343] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:30:33.876999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.877891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.877891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.879687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 682/1344] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:33.886286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a121d1-4db9-4848-b0f5-cf433470c6ac None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.887193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a121d1-4db9-4848-b0f5-cf433470c6ac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.887193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a121d1-4db9-4848-b0f5-cf433470c6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.980240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-11a121d1-4db9-4848-b0f5-cf433470c6ac None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:33.981964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-11a121d1-4db9-4848-b0f5-cf433470c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:34.117931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 663/1345] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:30:34.161248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.163255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:34.335236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.369300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ead5b-6b9b-4ecd-866d-deeca14795b5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.501817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.502932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.503404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.503865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.507096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:34.686922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.688461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b7df-497a-4ab6-bc74-6e97bc7ca0df service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.712053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 683/1346] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:30:39.692758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.693566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.693566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.694829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 664/1347] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:39.701348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb221-0d5f-49ac-9e3b-23fa8180b904 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.702844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb221-0d5f-49ac-9e3b-23fa8180b904 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.702844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb221-0d5f-49ac-9e3b-23fa8180b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.718929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.796897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-04dbb221-0d5f-49ac-9e3b-23fa8180b904 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:39.799268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-04dbb221-0d5f-49ac-9e3b-23fa8180b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:39.973050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 684/1348] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:30:40.041224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.044281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:40.195092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.316817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5afc6-3491-4248-8e54-77fa8b1753c9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.455551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.456189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.456466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.456740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.458561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:40.615110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.619526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.623741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c3ebc-2196-446e-a047-e1bc8cf6fac1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.652613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 665/1349] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:30:47.268787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ff75-8fc2-4651-9d42-4618eb1d0f36 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.269590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ff75-8fc2-4651-9d42-4618eb1d0f36 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.269590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ff75-8fc2-4651-9d42-4618eb1d0f36 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.270850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 685/1350] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:47.280387 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9dcf-97c7-449a-b618-99b60690b812 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.293045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9dcf-97c7-449a-b618-99b60690b812 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.298593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9dcf-97c7-449a-b618-99b60690b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.493672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c8be9dcf-97c7-449a-b618-99b60690b812 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:47.498085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8be9dcf-97c7-449a-b618-99b60690b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:47.833383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 666/1351] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:30:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:49.970756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=59 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.846926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35f344-1a92-4d4b-8aac-f628fddc0140 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.847660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35f344-1a92-4d4b-8aac-f628fddc0140 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.848282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35f344-1a92-4d4b-8aac-f628fddc0140 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.852009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 686/1352] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:53.857646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2b64-232c-4e7d-a940-831759888f12 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.858538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2b64-232c-4e7d-a940-831759888f12 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.858538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2b64-232c-4e7d-a940-831759888f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.990495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.039811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b3cf2b64-232c-4e7d-a940-831759888f12 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:54.041929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b3cf2b64-232c-4e7d-a940-831759888f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:54.282617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 667/1353] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:30:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:55.129276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786f2f-1427-4e5e-9da9-f46eee1a2f32 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.129982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786f2f-1427-4e5e-9da9-f46eee1a2f32 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.129982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786f2f-1427-4e5e-9da9-f46eee1a2f32 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.131795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 687/1354] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:55.137263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda714dd-7886-44e2-91b8-aa201f4f85c2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.137751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda714dd-7886-44e2-91b8-aa201f4f85c2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.137859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda714dd-7886-44e2-91b8-aa201f4f85c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.279166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fda714dd-7886-44e2-91b8-aa201f4f85c2 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:55.281710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fda714dd-7886-44e2-91b8-aa201f4f85c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:55.384462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.545754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 668/1355] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:59.160517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7a8-03ce-45e1-8c6a-3be2ca6f8a8a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.161340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7a8-03ce-45e1-8c6a-3be2ca6f8a8a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.161340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7a8-03ce-45e1-8c6a-3be2ca6f8a8a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.162545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 688/1356] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:59.170104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc211d-b346-4004-a547-aac034fcaced None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.170710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc211d-b346-4004-a547-aac034fcaced None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.171141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc211d-b346-4004-a547-aac034fcaced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.299001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bbdc211d-b346-4004-a547-aac034fcaced None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:59.302402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbdc211d-b346-4004-a547-aac034fcaced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:59.489694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 669/1357] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:59.583739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.584564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.585205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.589337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 689/1358] 10.222.0.77 () {54 vars in 962 bytes} [Mon Jul 18 16:30:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:59.609085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936eee8-dbd3-4451-9b7c-e81353ef5f72 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.610356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936eee8-dbd3-4451-9b7c-e81353ef5f72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.611075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936eee8-dbd3-4451-9b7c-e81353ef5f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.638246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.719496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1936eee8-dbd3-4451-9b7c-e81353ef5f72 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:59.720833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1936eee8-dbd3-4451-9b7c-e81353ef5f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:59.871988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 670/1359] 10.222.0.77 () {58 vars in 1059 bytes} [Mon Jul 18 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:30:59.881095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.882998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:00.074447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860eb1-d6b6-4889-84f1-d2f24fde691b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.204415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.267792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.268658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.269097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.269509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.271898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:00.392913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.395420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021c19-2c77-4b6f-87fd-0e6f2fdbc022 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.421352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 690/1360] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:02.864225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.865099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.865099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.866561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 671/1361] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:02.872493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5713-48a9-4ba3-8d8e-dcf3d71c5f99 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.873486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5713-48a9-4ba3-8d8e-dcf3d71c5f99 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.873486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5713-48a9-4ba3-8d8e-dcf3d71c5f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.961295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b80d5713-48a9-4ba3-8d8e-dcf3d71c5f99 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:02.962578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b80d5713-48a9-4ba3-8d8e-dcf3d71c5f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:03.018629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.102771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 691/1362] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:03.155768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.158708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:03.299165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.490485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea81d3d-e9c5-4620-a7ba-c1a941fde243 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.741055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.742729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.743324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.743946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.747768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:03.805780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.970955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.972705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a08bb2-5a2f-489e-80fb-f3ffa5f0936e service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.014737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 672/1363] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:06.956737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e0cec-b117-4e3b-a74d-e289401a8a8e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.957480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e0cec-b117-4e3b-a74d-e289401a8a8e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.958104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e0cec-b117-4e3b-a74d-e289401a8a8e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.961200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 692/1364] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:06.967975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06312-7498-4dd2-8e17-9bdc2db868ed None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.968685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06312-7498-4dd2-8e17-9bdc2db868ed None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.968685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06312-7498-4dd2-8e17-9bdc2db868ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.093274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-87f06312-7498-4dd2-8e17-9bdc2db868ed None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:07.095729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-87f06312-7498-4dd2-8e17-9bdc2db868ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:07.256902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:07.265492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 673/1365] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:07.341971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.342504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.343282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.346401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 693/1366] 10.222.0.77 () {54 vars in 962 bytes} [Mon Jul 18 16:31:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:07.357447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e0a8-6b5c-4e7d-8306-a28e3896cd58 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.358202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e0a8-6b5c-4e7d-8306-a28e3896cd58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.358837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e0a8-6b5c-4e7d-8306-a28e3896cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.523420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-06e4e0a8-6b5c-4e7d-8306-a28e3896cd58 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:07.525841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06e4e0a8-6b5c-4e7d-8306-a28e3896cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:07.796192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 674/1367] 10.222.0.77 () {58 vars in 1059 bytes} [Mon Jul 18 16:31:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:07.812536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:07.816864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:08.096808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d515071-b7b6-4e64-a8e9-92382ec6f3df None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:08.120614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:08.241450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:08.242120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.242393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.242765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.244477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:08.365542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:08.367724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c43e3-4f54-4b67-9b7a-b4495ea1b46f service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:08.391823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 694/1368] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:09.674890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e114246-e4e4-4671-82ee-f6a24cff1abd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.675632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e114246-e4e4-4671-82ee-f6a24cff1abd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.676285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e114246-e4e4-4671-82ee-f6a24cff1abd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.680386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 675/1369] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:09.686300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e087e-bead-44d0-b811-60c2e4898688 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.687153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e087e-bead-44d0-b811-60c2e4898688 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.687153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e087e-bead-44d0-b811-60c2e4898688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.729792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:09.777116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-695e087e-bead-44d0-b811-60c2e4898688 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:09.778557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-695e087e-bead-44d0-b811-60c2e4898688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:09.902772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 695/1370] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:10.218727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.219029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.219476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.220934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 676/1371] 10.222.0.77 () {54 vars in 964 bytes} [Mon Jul 18 16:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:10.226729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf602aa-27e5-455a-994f-555d8b82ae17 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.226917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf602aa-27e5-455a-994f-555d8b82ae17 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.227287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf602aa-27e5-455a-994f-555d8b82ae17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.317617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4bf602aa-27e5-455a-994f-555d8b82ae17 None None] MFA Rules not processed for user `527930459ed340c0b09ee6c6da770a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:10.318984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4bf602aa-27e5-455a-994f-555d8b82ae17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:10.383459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.426743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 696/1372] 10.222.0.77 () {58 vars in 1061 bytes} [Mon Jul 18 16:31:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:10.435563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.437127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:10.494446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.596611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75fc83-9a5d-4114-975e-c0104de62a2b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.709664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182042-9bbb-4c89-91f6-7d65eba94bcd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.710255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182042-9bbb-4c89-91f6-7d65eba94bcd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.710597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182042-9bbb-4c89-91f6-7d65eba94bcd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.712865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 697/1373] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:10.715626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.717669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.718602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.719071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.720893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fc45f-777c-4119-b3c8-bed57527963c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.721363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fc45f-777c-4119-b3c8-bed57527963c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.721830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fc45f-777c-4119-b3c8-bed57527963c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.722215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:10.823078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-374fc45f-777c-4119-b3c8-bed57527963c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:10.824433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-374fc45f-777c-4119-b3c8-bed57527963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:10.874176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.891400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.893998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.922186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 677/1374] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:10.947775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 698/1375] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:11.001851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.004093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:11.186250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c22ede-cd70-4b97-bc74-5fbfc0e881b2 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.325213 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.338540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.339196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.339474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.339743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.341522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:11.451317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.453128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2bab-4d29-475e-bb2e-6541a7b21e1f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.474587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 678/1376] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:14.473817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.475508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.475508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.477928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 699/1377] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:14.485312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ef637-5ea6-4edf-8afd-c052cc88e997 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.486146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ef637-5ea6-4edf-8afd-c052cc88e997 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.486350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ef637-5ea6-4edf-8afd-c052cc88e997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.591064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-211ef637-5ea6-4edf-8afd-c052cc88e997 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:14.592603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-211ef637-5ea6-4edf-8afd-c052cc88e997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:14.702472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.769782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 679/1378] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:14.820029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.822029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:14.885529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.995700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408412-e640-42ae-96db-f4dd97532e64 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.128843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.129516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.129801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.130076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.131874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:15.146032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.240225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.241950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f243-7c5b-4f46-a46d-97bf22160079 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.264822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 700/1379] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:17.499436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.501008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.501775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.504362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 680/1380] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:17.511543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef51e8-48f0-4fbc-9a15-7d75d9f1e94e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.512341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef51e8-48f0-4fbc-9a15-7d75d9f1e94e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.512700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef51e8-48f0-4fbc-9a15-7d75d9f1e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.606916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-48ef51e8-48f0-4fbc-9a15-7d75d9f1e94e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:17.608794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48ef51e8-48f0-4fbc-9a15-7d75d9f1e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:17.733694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:17.754564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 701/1381] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:17.805348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:17.807500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:18.046142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40251e-3bf6-406d-b4a6-9bff5daaf771 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:18.126865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:18.225241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:18.225901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.226168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.226484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.228193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:18.355324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:18.357194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bae6a-4903-4297-8116-91b58ba8d76e service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:18.381560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 681/1382] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:20.777633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.778163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.778163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.780195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 702/1383] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:20.786527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0feaf-9262-4bae-8645-10b8391b9f7c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.787068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0feaf-9262-4bae-8645-10b8391b9f7c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.787068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0feaf-9262-4bae-8645-10b8391b9f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.876629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-30c0feaf-9262-4bae-8645-10b8391b9f7c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:20.877865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-30c0feaf-9262-4bae-8645-10b8391b9f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:20.880507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:21.056866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 682/1384] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:21.155837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:21.157828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:21.351479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00093-2511-4d9f-8c0d-256f02357c71 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:21.440199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:21.524294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:21.524950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:21.525218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:21.525466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:21.527450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:21.701386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:21.703540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82dfb9-fb85-4681-8dd9-c3bafc542b0c service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.728575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 703/1385] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:31:24.136695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.138138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.138138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.140018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 683/1386] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:24.148051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbbca-911b-48d1-a9e9-d26c4ee3295c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.148773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbbca-911b-48d1-a9e9-d26c4ee3295c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.148773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbbca-911b-48d1-a9e9-d26c4ee3295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.251932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-906bbbca-911b-48d1-a9e9-d26c4ee3295c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:24.253606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-906bbbca-911b-48d1-a9e9-d26c4ee3295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:24.296288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.443266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 704/1387] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:24.496500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.498594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:24.718760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.741673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bf53a-ef3a-4a43-97c3-d1fb7238b519 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.943033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.944229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.944776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.945559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.949227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:25.109198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:25.112484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269fe88-cf51-4d6f-a6ef-d6dc26e1610c service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:25.123275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:25.147270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 684/1388] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:28.081243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.081837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.082229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.084793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 705/1389] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:28.093909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf150a-1e85-4b73-9592-593f9e97cd64 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.094833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf150a-1e85-4b73-9592-593f9e97cd64 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.095012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf150a-1e85-4b73-9592-593f9e97cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.182119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a9cf150a-1e85-4b73-9592-593f9e97cd64 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:28.183570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a9cf150a-1e85-4b73-9592-593f9e97cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:28.332423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 685/1390] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:28.410277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:28.413435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:28.486618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad230-c798-46a1-bdea-7c778b14936e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.487227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad230-c798-46a1-bdea-7c778b14936e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.487779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad230-c798-46a1-bdea-7c778b14936e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.490226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 686/1391] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:28.499573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172021d-66b0-4635-8651-638dd2d143a8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.500021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172021d-66b0-4635-8651-638dd2d143a8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.500455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172021d-66b0-4635-8651-638dd2d143a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.596024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b172021d-66b0-4635-8651-638dd2d143a8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:28.597586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b172021d-66b0-4635-8651-638dd2d143a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:28.627604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:28.677243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108551-2bb6-4f85-a7e3-eab0ecdb7ea5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:28.694387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:28.758017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 687/1392] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:28.830303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54906f-542d-42f2-987e-25cd7f62e1a4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.830925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54906f-542d-42f2-987e-25cd7f62e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.831292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54906f-542d-42f2-987e-25cd7f62e1a4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.833179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 688/1393] 10.222.0.77 () {54 vars in 961 bytes} [Mon Jul 18 16:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:28.839404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed3657-3d4f-444d-a80f-bf0d9acc2172 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.839853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed3657-3d4f-444d-a80f-bf0d9acc2172 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.840247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed3657-3d4f-444d-a80f-bf0d9acc2172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.848069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:28.849012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.849475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.849938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.853120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:28.968373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-70ed3657-3d4f-444d-a80f-bf0d9acc2172 None None] MFA Rules not processed for user `5e2880530a3d43b482fb27ac23701293`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:28.970165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70ed3657-3d4f-444d-a80f-bf0d9acc2172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:29.002294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.004302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.028741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 706/1394] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:29.082869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 689/1395] 10.222.0.77 () {58 vars in 1058 bytes} [Mon Jul 18 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:29.091246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.092519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:29.112411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.207056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dc1d3-98ff-4122-ad98-29cac49e81c5 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.336504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.338342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.338342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.338342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.340381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:29.480475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.513888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.515794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc846d7-16c2-4f60-874d-70a88f7796c9 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.539390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 707/1396] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:30.576862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6840ca-f244-4d68-bd96-76546605db04 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.577697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6840ca-f244-4d68-bd96-76546605db04 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.577978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6840ca-f244-4d68-bd96-76546605db04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.603683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:30.684033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-de6840ca-f244-4d68-bd96-76546605db04 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:30.686835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-de6840ca-f244-4d68-bd96-76546605db04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:30.825801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 690/1397] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:30.888371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.889256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.889256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.891049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 708/1398] 10.222.0.77 () {54 vars in 943 bytes} [Mon Jul 18 16:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:30.898675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c48-c947-432a-b039-e505c094ff36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.898964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c48-c947-432a-b039-e505c094ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.899241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c48-c947-432a-b039-e505c094ff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.024818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1b652c48-c947-432a-b039-e505c094ff36 None None] MFA Rules not processed for user `71aecafdbfad40d4979f6f6358a58109`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:31.026430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1b652c48-c947-432a-b039-e505c094ff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:31.098559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.218973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 691/1399] 10.222.0.77 () {58 vars in 1040 bytes} [Mon Jul 18 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:31.230174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.232600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:31.412058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2eed-5314-43ce-9d30-dfe3aa0f4a40 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.520445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.601592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.602446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.602885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.603283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.605550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:31.630756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505f7c-8079-4a39-92ef-e19c4853cd0f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.631337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505f7c-8079-4a39-92ef-e19c4853cd0f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.631337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505f7c-8079-4a39-92ef-e19c4853cd0f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.632746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 692/1400] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:31.639119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bde81e-ad3b-416f-bb36-184be7849674 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.639399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bde81e-ad3b-416f-bb36-184be7849674 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.639680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bde81e-ad3b-416f-bb36-184be7849674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.728165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-34bde81e-ad3b-416f-bb36-184be7849674 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:31.729417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-34bde81e-ad3b-416f-bb36-184be7849674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:31.742218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.744900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.756011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 709/1401] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:31:31.869913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 693/1402] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:31.917498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.918846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:32.038773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a693-86db-43eb-8797-5761e71beee7 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.050885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.211007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.211731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.212058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.212378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.214762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:32.361348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.362926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac01ce-7ba4-4e7c-9c15-8839188660dd service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.386652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 710/1403] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:31:34.661454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.662281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.662281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.667824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 694/1404] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:34.669643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dcc-2d87-41cd-880b-573760a22e33 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.670157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dcc-2d87-41cd-880b-573760a22e33 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.670271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dcc-2d87-41cd-880b-573760a22e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.691341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:34.759209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-58443dcc-2d87-41cd-880b-573760a22e33 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:34.760457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58443dcc-2d87-41cd-880b-573760a22e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:34.898616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 711/1405] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:34.952276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.953908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:34.977719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.170079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde485f1-c98d-48f6-89af-a4e2f6c29ffd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.296580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.343402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.344641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.345143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.345648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.348112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:35.489035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.491958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb16a8-87fe-44b2-b98f-dac0e033c185 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.528020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 695/1406] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:31:38.002110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.003020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.003020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.005292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 712/1407] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:38.017572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ee898-ab0a-497d-a130-e34fa503edbb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.017572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ee898-ab0a-497d-a130-e34fa503edbb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.017572 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ee898-ab0a-497d-a130-e34fa503edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.133570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-447ee898-ab0a-497d-a130-e34fa503edbb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:38.134842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-447ee898-ab0a-497d-a130-e34fa503edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:38.222396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.299833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 696/1408] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:31:38.348161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.349836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:38.507705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.625114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e8aa8-b853-45b1-8574-d37b17b3d491 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.875637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:38.876928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.877347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.877864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.881425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:38.994647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.044709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.046374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5e4f3-9456-485e-8863-e3185c4236f8 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.071889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 713/1409] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:31:48.297284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=19 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:49.068681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=21 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:57.366589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99b925-69a9-4b51-b2bb-c2a0da8c5f33 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:57.367199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99b925-69a9-4b51-b2bb-c2a0da8c5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:57.367357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99b925-69a9-4b51-b2bb-c2a0da8c5f33 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:57.368880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 697/1410] 10.222.0.77 () {54 vars in 918 bytes} [Mon Jul 18 16:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:57.375867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a18548-b083-49ab-9b54-f7a70b975815 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:57.376455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a18548-b083-49ab-9b54-f7a70b975815 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:57.376816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a18548-b083-49ab-9b54-f7a70b975815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:57.497134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90a18548-b083-49ab-9b54-f7a70b975815 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:57.499342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90a18548-b083-49ab-9b54-f7a70b975815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:57.708286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 714/1411] 10.222.0.77 () {58 vars in 1015 bytes} [Mon Jul 18 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:31:57.723789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83941e2-d2d6-4d13-b6da-e4e7e68929f8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:57.724090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83941e2-d2d6-4d13-b6da-e4e7e68929f8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:57.724334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83941e2-d2d6-4d13-b6da-e4e7e68929f8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:57.725752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 698/1412] 10.222.0.77 () {54 vars in 918 bytes} [Mon Jul 18 16:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:57.731607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91fe-f6ee-4f70-809f-38bf7b5b3f86 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:57.731969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91fe-f6ee-4f70-809f-38bf7b5b3f86 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:57.732239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91fe-f6ee-4f70-809f-38bf7b5b3f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:57.823465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:57.830197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c66c91fe-f6ee-4f70-809f-38bf7b5b3f86 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:57.832972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c66c91fe-f6ee-4f70-809f-38bf7b5b3f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:57.978498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 715/1413] 10.222.0.77 () {58 vars in 1015 bytes} [Mon Jul 18 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:00.659443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:00.660030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:00.660403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:00.663417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:00.670064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fe038-ee05-4b79-bf06-e36662d9a12a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:00.670709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fe038-ee05-4b79-bf06-e36662d9a12a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:00.670709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fe038-ee05-4b79-bf06-e36662d9a12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:00.751740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-602fe038-ee05-4b79-bf06-e36662d9a12a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:00.753013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-602fe038-ee05-4b79-bf06-e36662d9a12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:00.820886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:00.874205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:00.925971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:00.927571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:01.192674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b456bd3-e1f6-4324-a57b-eea4d8caf262 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:01.299529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:01.368042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:01.368704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:01.368987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:01.369259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:01.371059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:01.495924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:01.497813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c2811-1dfb-46b3-87d5-316492d0288d service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:01.518794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 700/1416] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:02.511477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673685-1076-4b29-a773-7324c984417b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:02.512364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673685-1076-4b29-a773-7324c984417b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:02.512364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb673685-1076-4b29-a773-7324c984417b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:02.513497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:02.519318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299636a-5da6-4560-a9bd-0e2c0b7cf4c9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:02.520192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299636a-5da6-4560-a9bd-0e2c0b7cf4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:02.520379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299636a-5da6-4560-a9bd-0e2c0b7cf4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:02.611396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d299636a-5da6-4560-a9bd-0e2c0b7cf4c9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:02.612660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d299636a-5da6-4560-a9bd-0e2c0b7cf4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:02.646244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:02.734005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:02.894102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb673685-1076-4b29-a773-7324c984417b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:02.895703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eb673685-1076-4b29-a773-7324c984417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:03.074330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb673685-1076-4b29-a773-7324c984417b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:03.125805 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:03.216328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:03.217214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:03.217627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:03.217949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:03.219825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:03.331327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:03.332977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046073f-1fa0-4e22-a0ea-9c8ee0a6bbd5 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:03.358242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 718/1419] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:04.353868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.354594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.354594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.356487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:04.363266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a693-8af4-46e7-aaf2-d5d293f54c06 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.363807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a693-8af4-46e7-aaf2-d5d293f54c06 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.364194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a693-8af4-46e7-aaf2-d5d293f54c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.455482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:04.498738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b406a693-8af4-46e7-aaf2-d5d293f54c06 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:04.500891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b406a693-8af4-46e7-aaf2-d5d293f54c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:04.651228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:04.807360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:04.809711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:05.013195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1498ea-c65b-42cc-99e3-a0338ab22ad2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:05.027794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:05.148806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:05.149470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:05.149760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:05.150039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:05.152364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:05.259691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:05.261638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ac23a-f869-48fd-b627-890793706d55 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:05.284190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 703/1422] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:06.206529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:06.207284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:06.207284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:06.208540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:06.214647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62396-372a-467a-9e43-e2439ef959e5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:06.215245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62396-372a-467a-9e43-e2439ef959e5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:06.215245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62396-372a-467a-9e43-e2439ef959e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:06.237162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:06.304096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-daa62396-372a-467a-9e43-e2439ef959e5 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:06.305373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-daa62396-372a-467a-9e43-e2439ef959e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:06.444774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:06.604559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:06.606075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:06.739127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:06.781807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a2cd-5832-4054-9b73-5168c68b53f0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:06.919521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:06.920183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:06.920467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:06.920744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:06.922553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:07.013485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:07.039654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:07.042850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ad68-e67d-4bec-aa6c-907231f10a47 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:07.079525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 721/1425] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:32:08.032008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:08.032621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:08.032621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.033986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:08.039953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217992f-f16d-4f19-bfb7-d2d2a35a6081 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:08.040539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217992f-f16d-4f19-bfb7-d2d2a35a6081 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:08.040539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217992f-f16d-4f19-bfb7-d2d2a35a6081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.131475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8217992f-f16d-4f19-bfb7-d2d2a35a6081 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:08.132861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8217992f-f16d-4f19-bfb7-d2d2a35a6081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:08.265871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:08.549453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:08.551045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:08.667890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:08.742344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1af9d5-18e5-4f9f-a80c-6b3b57bcf47f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:08.893811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:08.894367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:08.894684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:08.894960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.896790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:08.959380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:09.011940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:09.013613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cf6d4-d1ae-4dc3-9ec5-4f0d6f9eccf0 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:09.036389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 706/1428] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:32:09.897720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:09.898307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:09.898307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:09.899662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:09.905873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fbd72-8082-4f6b-8f14-6ca21ef4e573 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:09.906157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fbd72-8082-4f6b-8f14-6ca21ef4e573 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:09.906466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fbd72-8082-4f6b-8f14-6ca21ef4e573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:09.999105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3a2fbd72-8082-4f6b-8f14-6ca21ef4e573 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:10.000418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a2fbd72-8082-4f6b-8f14-6ca21ef4e573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:10.122147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:10.397236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:10.398971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:10.460158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:10.585337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274a2-e473-4fae-b01f-a782b7ef8f01 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:10.727044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:10.727709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:10.727994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:10.728268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:10.730135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:10.738622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:10.839265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:10.840912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c2045-8182-4f6f-82be-6ef5e47a75c3 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:10.863253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 724/1431] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:11.718727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:11.719559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:11.719758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:11.721315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:11.727479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1896b6-f563-41e9-9921-973cf0d2bff7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:11.728079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1896b6-f563-41e9-9921-973cf0d2bff7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:11.728079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1896b6-f563-41e9-9921-973cf0d2bff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:11.820021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bb1896b6-f563-41e9-9921-973cf0d2bff7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:11.821346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bb1896b6-f563-41e9-9921-973cf0d2bff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:11.902254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:11.938315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:12.096011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:12.097625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:12.118201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:12.277399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0df1a-d8f9-4d48-a9dc-6b7e134c18d7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:12.381121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:12.414624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:12.415443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:12.415831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:12.416210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:12.418106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:12.529567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:12.531300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae145b89-0d88-4f3b-97b5-c0d0450e030b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:12.553434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 709/1434] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:13.449412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:13.450054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:13.450054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:13.451517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:13.458287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912675-2c5a-476e-8fbb-0d47e073d850 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:13.458890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912675-2c5a-476e-8fbb-0d47e073d850 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:13.459220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912675-2c5a-476e-8fbb-0d47e073d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:13.562148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-19912675-2c5a-476e-8fbb-0d47e073d850 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:13.563476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-19912675-2c5a-476e-8fbb-0d47e073d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:13.619626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:13.693507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:13.851161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:13.852932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:14.023906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08536d95-e63e-4d76-8d09-a6f050054f37 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:14.097628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:14.169209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:14.169876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:14.170162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:14.170459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:14.172216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:14.279554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:14.281106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5d4e6-9d44-4517-a9ca-80dc491cb8aa service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:14.303810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 727/1437] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:32:15.159564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.160631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.160631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.161552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:15.168018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add930b-1d41-43b8-a04a-8ab8fea1e0ca None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.168604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add930b-1d41-43b8-a04a-8ab8fea1e0ca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.168604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add930b-1d41-43b8-a04a-8ab8fea1e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.315161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5add930b-1d41-43b8-a04a-8ab8fea1e0ca None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:15.317277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5add930b-1d41-43b8-a04a-8ab8fea1e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:15.321512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:15.437268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:15.597954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:15.599751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:15.771921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edca1a-9aa9-4115-9ea3-f837b26540dc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:15.804683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:15.910960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:15.911632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.911913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.912193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.913988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:16.027127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:16.028693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f1a8-e385-411f-85bb-0bc03b01e155 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:16.049847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 712/1440] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:18.094126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.095628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.095628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.096142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:18.102329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c7929-2aef-4daa-bb03-f9f97f37baef None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.103182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c7929-2aef-4daa-bb03-f9f97f37baef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.103182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c7929-2aef-4daa-bb03-f9f97f37baef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.153261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.183394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-566c7929-2aef-4daa-bb03-f9f97f37baef None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:18.184679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-566c7929-2aef-4daa-bb03-f9f97f37baef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:18.340498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:18.382818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:18.386176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:18.536317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba35c-cb67-4d3e-9879-7afb573453a5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:18.575612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.627066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:18.627789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.628172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.628569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.630581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:18.822476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:18.825572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efdda-838f-47cb-af51-0cc3fc4fa5a8 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:18.856534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 730/1443] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:21.260910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.261508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.261508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.263065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:21.269770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4fd2-6e17-4fda-8bce-c7a0adeaf68a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.270487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4fd2-6e17-4fda-8bce-c7a0adeaf68a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.270740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f4fd2-6e17-4fda-8bce-c7a0adeaf68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.355362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ee8f4fd2-6e17-4fda-8bce-c7a0adeaf68a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:21.356816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ee8f4fd2-6e17-4fda-8bce-c7a0adeaf68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:21.374962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:21.497320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:21.561955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:21.564232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:21.708242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1bb3b-d9ab-45a6-beab-b0e761d12fb5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:21.723674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:21.835592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:21.836629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.837184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.837955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.841855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:22.063581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:22.067126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db6a6-b6f8-44dc-aed7-1178bf7ba076 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:22.089278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 715/1446] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:24.961898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.962705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.962705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.964128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:24.972931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee38e-bade-4005-af82-9926f5738562 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.973492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee38e-bade-4005-af82-9926f5738562 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.973837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bee38e-bade-4005-af82-9926f5738562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:25.051812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:25.080109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f3bee38e-bade-4005-af82-9926f5738562 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:25.081496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3bee38e-bade-4005-af82-9926f5738562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:25.242772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:25.298576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:25.300412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:25.471304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a61ed-64f7-4e71-b216-c045d2a1e041 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:25.539286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:25.594302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:25.595305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:25.595781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:25.596198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:25.598223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:25.787991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:25.790824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99315d28-948b-4db3-8770-f520f19cf4e2 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:25.819869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 733/1449] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:28.362764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:28.363485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:28.363812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:28.366970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:28.375349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de318-f0c5-4ed8-adb7-eb63b1445719 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:28.376251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de318-f0c5-4ed8-adb7-eb63b1445719 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:28.376251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de318-f0c5-4ed8-adb7-eb63b1445719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:28.493602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2f1de318-f0c5-4ed8-adb7-eb63b1445719 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:28.494887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2f1de318-f0c5-4ed8-adb7-eb63b1445719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:28.519565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:28.652880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:28.717832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:28.720293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:28.902374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63781983-3ddf-44ba-9adf-85441ba85ddb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:28.928802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:29.026941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:29.027800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:29.028266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:29.028730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:29.031686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:29.202508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:29.203983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ffd0-b74a-4548-9a8a-2e5814012f87 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:29.232635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 718/1452] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:31.464762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:31.465509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:31.465509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:31.466803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:31.473101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba045c0-14c1-404a-ad82-3a18ac644074 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:31.474061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba045c0-14c1-404a-ad82-3a18ac644074 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:31.474061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba045c0-14c1-404a-ad82-3a18ac644074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:31.554015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:31.560015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ba045c0-14c1-404a-ad82-3a18ac644074 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:31.562543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ba045c0-14c1-404a-ad82-3a18ac644074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:31.707705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:31.747883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:31.749486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:31.875100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140f77b-15f4-4a9c-8597-69d07c2ea4b1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:31.933466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:31.963276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:31.963944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:31.964230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:31.964510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:31.966234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:32.112817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:32.114193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc9a8-a669-409f-a92b-b03b4ed2963e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:32.135339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 736/1455] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:34.348900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:34.349652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:34.349652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:34.350908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:34.357594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0882d70-659a-4cf1-828f-4c1f063b3696 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:34.358161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0882d70-659a-4cf1-828f-4c1f063b3696 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:34.358325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0882d70-659a-4cf1-828f-4c1f063b3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:34.442751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a0882d70-659a-4cf1-828f-4c1f063b3696 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:34.444643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a0882d70-659a-4cf1-828f-4c1f063b3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:34.484956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:34.577698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:34.644787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:34.646631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:34.770653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd23187-0659-4289-9174-7dcbd00ec876 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:34.806250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:34.864630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:34.865442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:34.865819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:34.866173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:34.868205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:35.001254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:35.002766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71b81a-ecc9-49c0-b30f-1aa8533b337f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:35.023631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 721/1458] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:32:37.844472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:37.845017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:37.845183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:37.846835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:37.853040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c0f0-7823-4d66-8c79-7352f500342b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:37.853958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c0f0-7823-4d66-8c79-7352f500342b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:37.853958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c0f0-7823-4d66-8c79-7352f500342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:37.932885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9638c0f0-7823-4d66-8c79-7352f500342b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:37.934110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9638c0f0-7823-4d66-8c79-7352f500342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:37.944326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:38.053486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:32:38.095726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:38.097190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:38.228407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a4123-0f15-4537-9199-32fd6a934b84 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:38.296228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:38.320515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:38.321177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:38.321489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:38.321846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:38.324146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:38.458716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:38.460299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839034-ff7f-42e3-bd77-bb041ef9c356 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:38.483838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 739/1461] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:40.662386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:40.662984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:40.662984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:40.664367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:40.670908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789a03-2235-4d2c-89ca-db33c1467f10 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:40.671705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789a03-2235-4d2c-89ca-db33c1467f10 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:40.671887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789a03-2235-4d2c-89ca-db33c1467f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:40.757098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-db789a03-2235-4d2c-89ca-db33c1467f10 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:40.759685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-db789a03-2235-4d2c-89ca-db33c1467f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:40.826369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:40.901417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:40.940332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:40.941895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:41.076640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d422e3-db5e-476d-905f-d7e2c42e6ecd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:41.122883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:41.168376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:41.169044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:41.169324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:41.169592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:41.171436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:41.306456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:41.308032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286eec30-c2a6-4523-896f-4e05ffd190d7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:41.329296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 724/1464] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:43.531574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:43.532150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:43.532150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:43.533549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:43.544091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6af44-ab9f-46da-a0e7-ff02a949a458 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:43.544755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6af44-ab9f-46da-a0e7-ff02a949a458 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:43.544900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6af44-ab9f-46da-a0e7-ff02a949a458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:43.640941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f0b6af44-ab9f-46da-a0e7-ff02a949a458 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:43.642281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f0b6af44-ab9f-46da-a0e7-ff02a949a458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:43.664594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:43.756663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:43.820100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:43.822431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:43.989880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a665-d674-48d1-b79a-1d2fabd29097 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:44.091105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:44.098830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:44.099500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:44.099782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:44.100058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:44.101785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:44.229253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:44.229808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae53dbe-1eb4-4486-9e42-d669e8712625 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:44.251015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 742/1467] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:32:46.401153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:46.401870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:46.401870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:46.403455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:32:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:32:46.410335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b0b7-31d0-4406-a0ab-7e128d53112b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:46.411030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b0b7-31d0-4406-a0ab-7e128d53112b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:46.411030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3b0b7-31d0-4406-a0ab-7e128d53112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:46.504346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-70d3b0b7-31d0-4406-a0ab-7e128d53112b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:46.505498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-70d3b0b7-31d0-4406-a0ab-7e128d53112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:46.563326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:46.626479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:32:46.665599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:46.667214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:46.814069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d97ff-54d2-4b6d-8a4e-746980e5df2d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:46.874651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:46.907796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:46.908487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:46.908762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:46.909032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:46.910874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:47.041439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:47.043011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8b393-81f1-4dc8-8711-b9e11d200dc7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:47.064220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 727/1470] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:32:56.625925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=22 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:57.061681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=66 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:08.335814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f397d55-d25a-4555-954b-c700f1b81f7b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:08.337171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f397d55-d25a-4555-954b-c700f1b81f7b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:08.337171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f397d55-d25a-4555-954b-c700f1b81f7b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:08.338901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 744/1471] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:08.345941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639cd7-1f0e-4891-93a1-aef8320665e3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:08.346581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639cd7-1f0e-4891-93a1-aef8320665e3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:08.346921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09639cd7-1f0e-4891-93a1-aef8320665e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:08.459221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-09639cd7-1f0e-4891-93a1-aef8320665e3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:08.460457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-09639cd7-1f0e-4891-93a1-aef8320665e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:08.572388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 728/1472] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:33:08.623634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:08.623924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:08.624207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:08.625611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 745/1473] 10.222.0.77 () {54 vars in 961 bytes} [Mon Jul 18 16:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:33:08.631304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223146-89d4-4d0f-bab7-0bbe56fddae1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:08.631649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223146-89d4-4d0f-bab7-0bbe56fddae1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:08.631978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223146-89d4-4d0f-bab7-0bbe56fddae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:08.723781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-66223146-89d4-4d0f-bab7-0bbe56fddae1 None None] MFA Rules not processed for user `5e2880530a3d43b482fb27ac23701293`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:08.725034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66223146-89d4-4d0f-bab7-0bbe56fddae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:08.750793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:08.830116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 729/1474] 10.222.0.77 () {58 vars in 1058 bytes} [Mon Jul 18 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:33:08.838106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:08.839675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:08.963013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1380a4-a051-40bc-87b1-de86807c7534 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:09.059470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:09.060547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:09.061037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:09.061505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:09.065035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:09.144127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:09.213331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:09.214804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818b08-0064-45f7-af1a-597ffecfcb72 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:09.237840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 746/1475] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:33:11.506765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:11.507408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:11.507408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:11.509526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 730/1476] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:33:11.515760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d80a1-5f00-4241-ba1f-1e52f9e507c4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:11.516391 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d80a1-5f00-4241-ba1f-1e52f9e507c4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:11.516524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d80a1-5f00-4241-ba1f-1e52f9e507c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:11.632234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f54d80a1-5f00-4241-ba1f-1e52f9e507c4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:11.634865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f54d80a1-5f00-4241-ba1f-1e52f9e507c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:11.786677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 747/1477] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:33:11.810992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:11.812619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:11.955639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f0c5-af36-4666-ac69-478ec87762ac None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:12.012449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:12.064957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:12.065537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:12.065820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:12.066094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:12.066872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:12.068999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:33:12.068999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:33:12.068999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:33:12.070163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79787a1c-d7c4-4e3d-a1a0-9a2512d8ef5c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:12.092664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 731/1478] 10.222.0.77 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:11 2022] GET /identity/v3/projects => generated 2553 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:33:14.260487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:14.261091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:14.261523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:14.264367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 748/1479] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:33:14.270805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41acbe-583d-42c6-8649-86cf7088c4d6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:14.271495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41acbe-583d-42c6-8649-86cf7088c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:14.271495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41acbe-583d-42c6-8649-86cf7088c4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:14.352481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8e41acbe-583d-42c6-8649-86cf7088c4d6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:14.353609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e41acbe-583d-42c6-8649-86cf7088c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:14.468749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 732/1480] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:33:14.516049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:14.518212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:14.529756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:14.726739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0ccbc-2619-45e5-b508-e74734fc1b26 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:14.821454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:14.866311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:14.867588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:14.867948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:14.868415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:14.870362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:14.989466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:14.991159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762bb2-b01b-495d-ac53-9f204371e81d service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:15.015506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 749/1481] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:33:17.354817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:17.355742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:17.355742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:17.357524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 733/1482] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:17.365350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853adb75-96d4-4e74-9f77-48dc4c4b8066 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:17.366199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853adb75-96d4-4e74-9f77-48dc4c4b8066 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:17.366394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853adb75-96d4-4e74-9f77-48dc4c4b8066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:17.497476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-853adb75-96d4-4e74-9f77-48dc4c4b8066 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:17.499658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-853adb75-96d4-4e74-9f77-48dc4c4b8066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:17.580910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:17.657927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 750/1483] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:33:17.722529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:17.725352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:17.771499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:17.927947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0e33e-7acd-46f3-90e8-f5ae78c22043 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:18.075712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:18.076744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:18.077250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:18.077751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:18.081020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:18.193249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:18.237386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:18.239366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff66226-bf21-4932-b14f-44a09423e011 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:18.262698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 734/1484] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:33:20.886956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:20.887859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:20.888038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:20.889608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 751/1485] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:33:20.896075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e2ab-5ac9-4da4-811c-46390b75e694 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:20.896690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e2ab-5ac9-4da4-811c-46390b75e694 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:20.896690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365e2ab-5ac9-4da4-811c-46390b75e694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:20.978022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7365e2ab-5ac9-4da4-811c-46390b75e694 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:20.979398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7365e2ab-5ac9-4da4-811c-46390b75e694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:21.103175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 735/1486] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:33:21.143223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:21.144825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:21.300778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355aec8-b535-4f1b-bd90-d45194b7e994 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:21.386652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:21.390926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:21.392230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:21.392766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:21.393519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:21.397243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:21.604463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:21.605961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0d634-c082-4bb1-8e20-bfd5eaf4b6e8 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:21.627914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 752/1487] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:33:24.012007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:24.012611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:24.012611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:24.014427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 736/1488] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:24.020689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615fa-3970-4e80-932b-57d721d30718 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:24.021264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615fa-3970-4e80-932b-57d721d30718 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:24.021264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615fa-3970-4e80-932b-57d721d30718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:24.100446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90a615fa-3970-4e80-932b-57d721d30718 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:24.101696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90a615fa-3970-4e80-932b-57d721d30718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:24.164079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:24.217304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 753/1489] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:33:24.261349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:24.264913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:24.307093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:24.389913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2aa2-32bb-419b-9e25-959691ab3423 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:24.476492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:24.477155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:24.477468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:24.477780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:24.479541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:24.596786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:24.632045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:24.633530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100ec1-00f4-4649-b8ea-1994fefa6a65 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:24.655880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 737/1490] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:33:27.040180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:27.040801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:27.040801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:27.042147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 754/1491] 10.222.0.77 () {54 vars in 926 bytes} [Mon Jul 18 16:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:27.048288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0547-c3a7-4653-b89a-02c895b7470c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:27.048875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0547-c3a7-4653-b89a-02c895b7470c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:27.048875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0547-c3a7-4653-b89a-02c895b7470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:27.127714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e18b0547-c3a7-4653-b89a-02c895b7470c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:27.128984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e18b0547-c3a7-4653-b89a-02c895b7470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:27.249698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 738/1492] 10.222.0.77 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:33:27.292478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:27.294076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:27.417116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7d696-79a6-4e95-8b8c-8bb318ee547a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:27.505409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:27.506082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:27.506369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:27.506681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:27.508520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:27.519572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:27.719591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:27.722734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18602c9-bc0a-4bad-9829-6e7c0969a819 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:27.754543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 755/1493] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:33:37.247587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=82 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:37.751943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=50 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:35:02.565891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:02.567070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:02.567070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:02.682961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:35:02.684353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:02.809324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 739/1494] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:35:02.819187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:02.819830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:02.819830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:02.841308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:35:02.842624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:02.852165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 756/1495] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:35:02.857566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:02.867762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7908a0-22b0-4e55-bd66-b349b7c2f396 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:02.872646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:02.873320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:02.873601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:02.873873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:02.874674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:35:02.874981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:02.876029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:02.876029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:35:02.876029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:35:02.876946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:02.890924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 740/1496] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:35:02.900423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:02.910627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f843c3c-4ef3-4bc4-90b8-17c87bc879d0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:02.915514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:02.916217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:02.916504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:02.916783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:02.917505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:02.919400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:02.919400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:35:02.919400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:35:02.920628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:02.966093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 757/1497] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:35:02 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:35:02.972333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:02.988923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d82df-eff1-4087-b9cb-9daccdfdc507 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.078939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:35:03.087685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.088515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.088919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.089297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.090099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.091371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:03.091371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:35:03.091371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:35:03.093001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.116068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 741/1498] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:35:02 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:35:03.120438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:03.133149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389c64-fb36-4928-bf57-6f5928373298 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.139063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.139891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.140224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.140549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.141390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.142667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:03.142667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:35:03.142667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:35:03.144897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.163057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 758/1499] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:35:03.167866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:03.179399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d104ed-2c8d-45c1-8d0c-593d44a1d193 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.184742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.185462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.185745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.186029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.186818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.187853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:03.187853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:35:03.187853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:35:03.188780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39332482-f132-4105-bb33-0568d621a4d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.205963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 742/1500] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:35:03.210596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:03.222205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdaf6c-8cb5-4a90-8a0f-02ec9bb8d4e5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.228159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.229161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.229488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.229811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] PATH_INFO: `/v3/projects/adbdade3b4774010b61df0da88298e28/users/5bcc3fbd61a84aeca5273a581a6dd405/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.249502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbdade3b4774010b61df0da88298e28, user_id=5bcc3fbd61a84aeca5273a581a6dd405, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.250609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:03.250609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:35:03.250609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:35:03.252589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826ca6d-8cde-4215-9418-e39682690fdd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.269474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 759/1501] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:35:03 2022] PUT /identity/v3/projects/adbdade3b4774010b61df0da88298e28/users/5bcc3fbd61a84aeca5273a581a6dd405/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:35:03.282386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217ac0-c7d5-41f5-8568-69b952b85171 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.282741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217ac0-c7d5-41f5-8568-69b952b85171 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.282929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00217ac0-c7d5-41f5-8568-69b952b85171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.313545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-00217ac0-c7d5-41f5-8568-69b952b85171 None None] MFA Rules not processed for user `5bcc3fbd61a84aeca5273a581a6dd405`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:35:03.315113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-00217ac0-c7d5-41f5-8568-69b952b85171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:03.392499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 743/1502] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:35:03.408038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d018-6605-4aa7-b785-d2530a597add None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.408384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d018-6605-4aa7-b785-d2530a597add None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.408718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d018-6605-4aa7-b785-d2530a597add None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.410218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 760/1503] 10.222.0.77 () {54 vars in 964 bytes} [Mon Jul 18 16:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:35:03.416044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fea63-f40e-4918-b914-0f344a528785 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.416380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fea63-f40e-4918-b914-0f344a528785 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.416803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fea63-f40e-4918-b914-0f344a528785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.485869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-824fea63-f40e-4918-b914-0f344a528785 None None] MFA Rules not processed for user `527930459ed340c0b09ee6c6da770a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:35:03.487685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-824fea63-f40e-4918-b914-0f344a528785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:03.581754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 744/1504] 10.222.0.77 () {58 vars in 1061 bytes} [Mon Jul 18 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:35:03.590570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5d018-6605-4aa7-b785-d2530a597add None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:03.609584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5d018-6605-4aa7-b785-d2530a597add None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.616441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.617634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.618150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.618695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.634512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.636008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.648746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 761/1505] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:35:03.700373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ce7ed-b325-4b44-ab29-50d7927a0b61 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.700891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ce7ed-b325-4b44-ab29-50d7927a0b61 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.701353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ce7ed-b325-4b44-ab29-50d7927a0b61 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.703796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 745/1506] 10.222.0.77 () {52 vars in 829 bytes} [Mon Jul 18 16:35:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:35:03.708331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:03.780422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff570-3afa-4cea-a831-ee6e585b4cf0 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:03.788299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:03.789846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.790353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.790907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] PATH_INFO: `/v3/users/5bcc3fbd61a84aeca5273a581a6dd405` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:03.793166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcc3fbd61a84aeca5273a581a6dd405)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:03.795173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:03.795173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:35:03.795173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:35:03.797369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:03.903642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcc3fbd61a84aeca5273a581a6dd405'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:03.928021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcc3fbd61a84aeca5273a581a6dd405'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:03.953392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcc3fbd61a84aeca5273a581a6dd405'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:03.958781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:35:03.969796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcc3fbd61a84aeca5273a581a6dd405'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:03.970763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcc3fbd61a84aeca5273a581a6dd405'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:03.980926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 762/1507] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:35:03 2022] DELETE /identity/v3/users/5bcc3fbd61a84aeca5273a581a6dd405 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:35:03.985848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:03.986171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:03.986448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:04.005588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:35:04.006855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:04.017933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:35:04.076178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 746/1508] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:35:04.088167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:04.089683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:04.182243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ab425-f0bf-4166-9250-55ddf6eba63b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:04.186624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:04.187402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:04.187726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:04.188041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:04.199331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:04.201530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee68b7-c52d-462f-9209-c9c68c047629 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:04.214009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 763/1509] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:35:04.672914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:35:04.675794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:35:04.691999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c736b-9480-4e97-86ad-c88e6e857cb9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:35:04.697020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:35:04.698109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:35:04.698529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:35:04.698875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] PATH_INFO: `/v3/projects/adbdade3b4774010b61df0da88298e28` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:35:04.700311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbdade3b4774010b61df0da88298e28)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:35:04.701479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:35:04.701479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:35:04.701479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:35:04.702917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:35:04.779152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbdade3b4774010b61df0da88298e28', 'request_id': 'req-c74b058e-7dc3-4b88-b432-1e741b2fd149'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:04.791823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbdade3b4774010b61df0da88298e28', 'request_id': 'req-c74b058e-7dc3-4b88-b432-1e741b2fd149'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:04.808873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbdade3b4774010b61df0da88298e28', 'request_id': 'req-c74b058e-7dc3-4b88-b432-1e741b2fd149'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:04.823180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] The token cache is being invalidate because project adbdade3b4774010b61df0da88298e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:35:04.823537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:04.824514 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c74b058e-7dc3-4b88-b432-1e741b2fd149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:35:04.826288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 747/1510] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:35:04 2022] DELETE /identity/v3/projects/adbdade3b4774010b61df0da88298e28 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:35:14.209003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:35:14.819440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:51.556346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.557052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.557230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.558677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:51.566265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a7886-930b-4b2e-bd63-b85e87beeddf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.566915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a7886-930b-4b2e-bd63-b85e87beeddf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.567282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a7886-930b-4b2e-bd63-b85e87beeddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.597772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f96a7886-930b-4b2e-bd63-b85e87beeddf None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:51.599779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f96a7886-930b-4b2e-bd63-b85e87beeddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:51.688916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:51.852994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:36:51.854972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:51.979772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ebc13-4d4e-4fb7-9208-d0d50e27cd2b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:36:51.986084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:36:51.986791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.987079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.987361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.997144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:36:51.998947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3884437-09a0-4a10-a694-12942bfce29d service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:36:52.010777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 765/1513] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:36:52.819857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71109-d822-4561-b913-e033e77d3d28 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:52.820695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71109-d822-4561-b913-e033e77d3d28 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:52.820695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71109-d822-4561-b913-e033e77d3d28 None None] PATH_INFO: `/v3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:52.822193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 749/1514] 10.222.0.95 () {54 vars in 925 bytes} [Mon Jul 18 16:36:52 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:36:52.829224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2dd0-b84b-4769-9e2c-e5b8669eeba6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:52.830066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2dd0-b84b-4769-9e2c-e5b8669eeba6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:52.830317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2dd0-b84b-4769-9e2c-e5b8669eeba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:52.929320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcd7-c8a6-44cc-a4c4-4722451432e9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:52.929709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcd7-c8a6-44cc-a4c4-4722451432e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:52.930137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcd7-c8a6-44cc-a4c4-4722451432e9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:52.932103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 750/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:36:52.939077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795561ba-8046-4290-9e35-9950a359e470 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:52.939585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795561ba-8046-4290-9e35-9950a359e470 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:52.939939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795561ba-8046-4290-9e35-9950a359e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:53.028713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bdda2dd0-b84b-4769-9e2c-e5b8669eeba6 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:53.030643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bdda2dd0-b84b-4769-9e2c-e5b8669eeba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:53.090210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-795561ba-8046-4290-9e35-9950a359e470 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:53.092845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-795561ba-8046-4290-9e35-9950a359e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:53.106941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 751/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:36:53.123964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 766/1517] 10.222.0.95 () {58 vars in 1015 bytes} [Mon Jul 18 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:36:53.390433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795561ba-8046-4290-9e35-9950a359e470 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:36:53.412392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795561ba-8046-4290-9e35-9950a359e470 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:36:53.421678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:36:53.422826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:53.423305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:53.423773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:53.439528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:36:53.442628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dda75d-2ffd-4f6c-976f-13a0d3daeb78 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:36:53.461646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 752/1518] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:02.274075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0c87-8207-465e-9ea3-facb96ca13e5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:02.274796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0c87-8207-465e-9ea3-facb96ca13e5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:02.274796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0c87-8207-465e-9ea3-facb96ca13e5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:02.275952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 767/1519] 10.222.0.77 () {54 vars in 961 bytes} [Mon Jul 18 16:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:37:02.282863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf2903-6478-40c7-9555-f433df6642f7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:02.283418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf2903-6478-40c7-9555-f433df6642f7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:02.283857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf2903-6478-40c7-9555-f433df6642f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:02.308617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c2cf2903-6478-40c7-9555-f433df6642f7 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:02.309859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c2cf2903-6478-40c7-9555-f433df6642f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:02.320355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 753/1520] 10.222.0.77 () {58 vars in 1058 bytes} [Mon Jul 18 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:02.329600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db0c87-8207-465e-9ea3-facb96ca13e5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:02.343862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db0c87-8207-465e-9ea3-facb96ca13e5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:02.349086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:02.349757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:02.350039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:02.350313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:02.360349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:02.362026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4ecee-07f7-424c-a101-32cea3a1f368 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:02.374013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 768/1521] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:37:04.732852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbdbef-970c-4175-94ae-a289b3e44fa7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:04.733436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbdbef-970c-4175-94ae-a289b3e44fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:04.733532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbdbef-970c-4175-94ae-a289b3e44fa7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:04.734921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:04.740458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41ddca-ebc7-4b8f-9921-e4047c0265c2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:04.741013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41ddca-ebc7-4b8f-9921-e4047c0265c2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:04.741013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41ddca-ebc7-4b8f-9921-e4047c0265c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:04.761892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2a41ddca-ebc7-4b8f-9921-e4047c0265c2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:04.763769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a41ddca-ebc7-4b8f-9921-e4047c0265c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:04.773079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:04.937738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbdbef-970c-4175-94ae-a289b3e44fa7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:04.954328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbdbef-970c-4175-94ae-a289b3e44fa7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:04.960241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:04.960917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:04.961208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:04.961488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:04.971625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:04.973278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf8d1a-d81d-45ed-a551-1260fcf8c48f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:04.985024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 755/1524] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:05.849339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf6e62-a857-40fe-b9ae-52da545762af None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:05.849951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf6e62-a857-40fe-b9ae-52da545762af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:05.849951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf6e62-a857-40fe-b9ae-52da545762af None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:05.851310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:05.857502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5627c-48df-44d0-a670-5e9f19f552fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:05.858137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5627c-48df-44d0-a670-5e9f19f552fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:05.858137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5627c-48df-44d0-a670-5e9f19f552fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:05.877516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-34e5627c-48df-44d0-a670-5e9f19f552fc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:05.878794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-34e5627c-48df-44d0-a670-5e9f19f552fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:05.887153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:06.048181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf6e62-a857-40fe-b9ae-52da545762af None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:06.056803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:06.066282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf6e62-a857-40fe-b9ae-52da545762af None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:06.072316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:06.073002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:06.073285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:06.073556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:06.084066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:06.086163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a19dd-ef20-459e-979f-89e0e94f9dd1 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:06.097670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 771/1527] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:10.137579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:10.139106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:10.139266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:10.141183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:37:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:10.149752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3623c6-c2c8-4812-bfc1-126be96d5bb7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:10.150537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3623c6-c2c8-4812-bfc1-126be96d5bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:10.150776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3623c6-c2c8-4812-bfc1-126be96d5bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:10.172554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ed3623c6-c2c8-4812-bfc1-126be96d5bb7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:10.173920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed3623c6-c2c8-4812-bfc1-126be96d5bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:10.183277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:10.224454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:10.226394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:10.295952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa37c4-3e75-459d-a470-be4253c5ac87 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:10.300116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:10.300817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:10.301092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:10.301363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:10.312441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:10.313904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d75aa-311e-47be-b0a9-3df245100acb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:10.326366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 758/1530] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:12.451428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9503-b57c-475b-b30e-7c09a186c719 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:12.452281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9503-b57c-475b-b30e-7c09a186c719 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:12.452281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec9503-b57c-475b-b30e-7c09a186c719 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:12.453394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:37:12.461729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9875fcc-7fa3-4016-ab73-8a1532b1be31 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:12.462319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9875fcc-7fa3-4016-ab73-8a1532b1be31 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:12.462319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9875fcc-7fa3-4016-ab73-8a1532b1be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:12.483475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b9875fcc-7fa3-4016-ab73-8a1532b1be31 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:12.484714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b9875fcc-7fa3-4016-ab73-8a1532b1be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:12.497174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:12.534731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec9503-b57c-475b-b30e-7c09a186c719 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:12.545974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec9503-b57c-475b-b30e-7c09a186c719 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:12.549799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:12.550519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:12.550806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:12.551082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:12.562082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:12.563598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316b50-9fa0-4a85-a643-415958be306a service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:12.574502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 774/1533] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:16.218995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:16.219816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:16.219816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:16.220976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:37:16.227137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5f5e1-df63-4575-96fd-6491204c8c13 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:16.227969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5f5e1-df63-4575-96fd-6491204c8c13 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:16.227969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5f5e1-df63-4575-96fd-6491204c8c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:16.247823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c0e5f5e1-df63-4575-96fd-6491204c8c13 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:16.249074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c0e5f5e1-df63-4575-96fd-6491204c8c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:16.257332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:16.586606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:16.588610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:16.597635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=78 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:16.661810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1c09b-ee83-47e2-890a-92b230d48fa7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:16.665721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:16.666383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:16.666692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:16.666960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:16.677909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:16.679536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cf3e4-5ed4-4d37-bffc-ff3157c90529 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:16.691364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 761/1536] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:37:18.905613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:18.906199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:18.906199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:18.907530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:18.915474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4e39-72da-4eb1-a4f3-59269f15cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:18.916029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4e39-72da-4eb1-a4f3-59269f15cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:18.916329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4e39-72da-4eb1-a4f3-59269f15cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:18.943043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02ce4e39-72da-4eb1-a4f3-59269f15cfa7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:18.945148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02ce4e39-72da-4eb1-a4f3-59269f15cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:18.959362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:19.287762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:19.289494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:19.303653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f6630-fa8b-4c50-9e89-ebc624b160f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:19.307486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:19.308157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:19.308434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:19.308721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:19.320337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:19.321803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481111ec-87e9-4456-bb14-0872a5d5c870 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:19.333337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 777/1539] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:28.935418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=28 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:29.328643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=23 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:56.798308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:56.800393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:56.801281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:56.844522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:56.847216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:56.871988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 763/1540] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:56.882881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:56.884500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:56.885281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:56.943795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:56.946001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:56.974509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 778/1541] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:56.980349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.011428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e732-3e6f-4013-b419-c6d612721fb5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.024403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.025588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.026090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.026631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.028069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:57.028593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.031819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3b53d-d642-4e80-a7da-ee7420c2e4d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.069911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 764/1542] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:57.081704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.108304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537122a6-53eb-48ee-8f3f-2add82510189 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.122074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.123184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.123408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e151c9-7b98-459c-bf88-313c80a2eb2f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.123755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e151c9-7b98-459c-bf88-313c80a2eb2f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.123936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.124216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e151c9-7b98-459c-bf88-313c80a2eb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.124387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.125860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.129623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.178577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d6e151c9-7b98-459c-bf88-313c80a2eb2f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.181194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6e151c9-7b98-459c-bf88-313c80a2eb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.213060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 779/1543] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:57.218403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.220728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 765/1544] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.243907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2931-2054-4ae0-a7cc-ec80bbe36dca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.244359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2931-2054-4ae0-a7cc-ec80bbe36dca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.244811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2931-2054-4ae0-a7cc-ec80bbe36dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.317890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ae0d2931-2054-4ae0-a7cc-ec80bbe36dca None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.322682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ae0d2931-2054-4ae0-a7cc-ec80bbe36dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.383835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7056ee-7e17-442c-8376-859a14431d04 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.415875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.417071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.417560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.418067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.419448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.421428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:38:57.421428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:38:57.421428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:38:57.433807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08d432-97c2-4c77-90e8-455dc100fed6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.501179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 780/1545] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:38:57.503695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.504158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.504605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.554949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.557895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.574734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 766/1546] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.576231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.576516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.576843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.592005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 781/1547] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.593394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.615400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.616648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.634699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830250-5730-44db-98b4-95cc1100cd6f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.639967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 767/1548] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.641023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.661005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.662702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.663056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.670502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.670502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.670933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f1dc3-f1ce-440a-bda3-89d62ca38ee8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.675397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc3199-b7d2-476f-b240-6c61cb8e77da admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.691705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.693112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.693622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.694111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.695697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:57.696266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.708076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961c025-31eb-4ce1-a56f-eac7e75fc7e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.718944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 782/1549] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:57.725162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.725646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.726196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.747376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 768/1550] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:57.749700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.750211 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.769143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.805963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.808438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.834773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 783/1551] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.836891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.837101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:57.839727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.857688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078eb8ae-1d70-46d2-baba-5d40da138010 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.865796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.866907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.867278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.867598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.868359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.870035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.875493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 769/1552] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.876933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.891682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 784/1553] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:57.896333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.926727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ace47-726a-45b5-a1c3-49d4234adf58 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.927966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042132d6-65bf-496d-ae48-a0cbaa7ea75f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.944715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.947006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.948117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.948605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.949101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.950519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:57.951076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.952607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.953104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.953587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.954933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.957305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:38:57.957305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:38:57.957305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:38:57.959653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.962945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:38:57.962945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:38:57.962945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:38:57.964223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.991528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 785/1554] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:57.993653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.019530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47184ad-1c56-4e45-8f33-809040fce580 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.023742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 770/1555] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:57 2022] POST /identity/v3/projects => generated 393 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.030791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.127972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.129470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.129926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.130399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] PATH_INFO: `/v3/projects/26a41505f0ce4a66a02a5d954da278cb/users/4909d4aef302404290adcd3268e53c1b/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.147839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a41505f0ce4a66a02a5d954da278cb, user_id=4909d4aef302404290adcd3268e53c1b, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.151157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dca46-8252-4547-9846-cce7ff9f66b2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.154735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.160295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5baf-37df-4c0d-90e0-e6f7d011faf8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.167775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 786/1556] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:38:57 2022] PUT /identity/v3/projects/26a41505f0ce4a66a02a5d954da278cb/users/4909d4aef302404290adcd3268e53c1b/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:38:58.169193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc98be-79f4-4f78-9be4-b59ececb53b1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.169491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc98be-79f4-4f78-9be4-b59ececb53b1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.169768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc98be-79f4-4f78-9be4-b59ececb53b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.189827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-13dc98be-79f4-4f78-9be4-b59ececb53b1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.191129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-13dc98be-79f4-4f78-9be4-b59ececb53b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.219539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.271097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.272835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.272835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.273173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.274925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 787/1557] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.282999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:58.282999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.283337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.283337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.283337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.283791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.300432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 771/1558] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:58.301936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.317541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.318212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad41962-c187-4950-832a-a4c0140ce5b8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.319820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.324294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.325304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.325737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.326155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.327252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.329425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10173d-3513-454e-831e-b608b6c0d0df admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.334504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 788/1559] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.342047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.353750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aace2-ee7b-489a-898d-cae6644cf4c6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.362535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.363657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.364157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.364647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.365526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 772/1560] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.369758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.369866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d797f-e712-4bac-a402-52e67bbf94ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.369866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d797f-e712-4bac-a402-52e67bbf94ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.369866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d797f-e712-4bac-a402-52e67bbf94ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.370332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12553eec-0bfc-4b17-81ed-0478b9463d37 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.388393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db2d797f-e712-4bac-a402-52e67bbf94ab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.389586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db2d797f-e712-4bac-a402-52e67bbf94ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.403390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 789/1561] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/users => generated 297 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.404908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.405271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.405622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.426658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.427919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.482542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 773/1562] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.482542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a19dd3-acb3-411b-97b9-7680d5084496 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.482542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a19dd3-acb3-411b-97b9-7680d5084496 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.482542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a19dd3-acb3-411b-97b9-7680d5084496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.500172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-05a19dd3-acb3-411b-97b9-7680d5084496 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.501410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-05a19dd3-acb3-411b-97b9-7680d5084496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.510288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 774/1563] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.511955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7ae99-db43-40ba-8300-b45fb768d553 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.512244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7ae99-db43-40ba-8300-b45fb768d553 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.512530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7ae99-db43-40ba-8300-b45fb768d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.530474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 790/1564] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.530474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.536241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e8a7ae99-db43-40ba-8300-b45fb768d553 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.538540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e8a7ae99-db43-40ba-8300-b45fb768d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.545306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3d5d-808c-4c2b-9049-adaa2b20faa4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.548603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 775/1565] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.550701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.551473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.551835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.552182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.553089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.553279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.553951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.554439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.556023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.577791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.579635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.591184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 791/1566] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.592988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.599024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 776/1567] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.600339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.613345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861858-b99a-42b9-9f32-c473c75177bc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.672627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.695710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc2482-de51-4a20-a24b-042f52ce27e7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.702852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.703796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.703958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.704439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.704900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.705011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.705290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.705575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.706233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.706943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.709345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948ba8f-908d-4eea-8642-c979eda6725d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.710305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.729473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 777/1568] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:58.731638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511da5-f03b-4e89-8422-38e7e61185be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.732139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511da5-f03b-4e89-8422-38e7e61185be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.732675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511da5-f03b-4e89-8422-38e7e61185be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.738923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 792/1569] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.743869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.764544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85511da5-f03b-4e89-8422-38e7e61185be None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.766389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85511da5-f03b-4e89-8422-38e7e61185be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.780952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 778/1570] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.783677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.784174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.784732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.809657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.810949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.811960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cc72-c3c4-40f7-9b9c-797e8155ea82 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.818329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.819115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 779/1571] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.819491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.819963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.820150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.820564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.830494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db4b58-0019-4f42-b0c6-7cba0737be00 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.835493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.836227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.836516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.836788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.837410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.837581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:58.838214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.839479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.840200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.842445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.858124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 793/1572] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:58.859559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.863606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 780/1573] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:58.864411 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.874601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5e146-df9b-4dc6-87bd-d2c7bf5e2c62 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.875593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab5bf-1ba3-4e87-a7e0-e8366431628e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.881171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.881553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.882244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.882461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.882668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.882940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.883267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.883704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:58.883910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.884090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.885045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.885665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.888148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.901079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 781/1574] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:58.904090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.919847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415acc8b-c02a-471f-9a66-b6e2f75ba91c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.926344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 794/1575] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/users => generated 286 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:58.927744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.929500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.930996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.931286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.931618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.932640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.934499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.942926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9376dd-c4e7-41cb-832d-765a33790339 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.949066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 782/1576] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:58.949537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.950316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.950758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.951201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.952487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.953208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.954757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.968267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a516-37a9-4859-97e1-93497437003d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.971608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 795/1577] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:58.973336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.982372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.983785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.984036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.984400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.986064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:58.986701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:58.989127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:58.996547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e106b5-7280-4f9b-ae06-8627e937e034 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.006573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.007781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.008374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.008923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.011105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:38:59.012266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.016034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.026869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 783/1578] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:59.026869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.036141 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33597d9-9479-4595-a7f5-c9297eea5178 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.037658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 796/1579] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:38:59.038782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.053550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.054296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.054763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.055102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.055870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.058166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.059923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31acda-560b-4fc9-979b-0c1085821b47 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.065685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.066775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.067207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.067531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.068538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.071398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.106459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 784/1580] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.106713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.113929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 797/1581] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.118266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.215352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04736b-0eed-48f5-b3fa-41e4794c1130 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.220999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.222777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.223302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.223843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] PATH_INFO: `/v3/projects/afb65421f9504a918b7a47451fac6707/users/ab3be59ea4c74da3b4e19273102bccd9/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.233884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa423-32fe-47b2-82f6-ab532c9e078d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.240664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.241497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.241785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.242255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.243794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.246320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.253959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb65421f9504a918b7a47451fac6707, user_id=ab3be59ea4c74da3b4e19273102bccd9, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.258351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 785/1582] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:59.260566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.265626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.277702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef33b-1322-49a4-9a4e-5221909f90eb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.289110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 798/1583] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:38:59 2022] PUT /identity/v3/projects/afb65421f9504a918b7a47451fac6707/users/ab3be59ea4c74da3b4e19273102bccd9/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:38:59.291268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.299123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:59.363719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.364429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.364747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.365073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.365763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.367729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.386140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 786/1584] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:59.386140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.394858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7be7f-f5a8-45a6-a556-95c4fdec876d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.402201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.402575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.402737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.403045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.403624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.405608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.437599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1be18-7ca2-4452-a224-2dc7c5fc0e18 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.448309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.449481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.450227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.450758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.452130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.456077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.460164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 787/1585] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/projects => generated 405 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.469756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.496089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:59.530048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 799/1586] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.534879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.599420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8c909-b46a-459b-9e66-36b6e2c0bead admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.661191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613d99-bb6f-4245-9b6a-0198dcb30915 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.667458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.668416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.668883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.669322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.670184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.672611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.699490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.700177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.700462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.700737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.701763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.704337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06bb0e-e590-4bf5-a490-d4cc5a9fbb00 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.714609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 800/1587] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.715557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.728740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed36167-477a-468d-8a08-0b8b42bdac7f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.734887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.735706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.736028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.736318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.737269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.739183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.747889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 788/1588] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:38:59.747889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.747889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.747889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.751218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 801/1589] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:38:59.752090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.769181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa406b9-1480-4099-9a06-34ebb1efa82c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.776482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.777444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.777681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.777957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] PATH_INFO: `/v3/projects/8ff24b729d5747ca8bc7c6bb5e8293c1/users/2b4ce342ddf14eef97b7c112a2bdd387/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.786471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:59.787979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:59.798310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 789/1590] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:59.803938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.804295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff24b729d5747ca8bc7c6bb5e8293c1, user_id=2b4ce342ddf14eef97b7c112a2bdd387, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.804295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.830385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 802/1591] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:38:59 2022] PUT /identity/v3/projects/8ff24b729d5747ca8bc7c6bb5e8293c1/users/2b4ce342ddf14eef97b7c112a2bdd387/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:38:59.831980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:59.835393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db663d29-bcc9-4902-ac9f-88adb83a6622 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.924779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:59.956466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.959461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.960140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.960611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.961780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.964517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.966988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf664-0148-470c-b90f-1eabf42569ca admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:59.977048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:59.978362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:59.979072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:59.979667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:59.981617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.985955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.003380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 790/1592] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/users => generated 303 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:00.004748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.019752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c64c3-0d36-4046-8fa1-251d84bbe93d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.027597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.028712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.029239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.029744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 803/1593] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:38:59 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:00.030490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.031709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.031838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.033758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.047130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 791/1594] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.048408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.060294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77e9c7-e233-41dd-ab46-8ea5fa5eeb39 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.064095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8384530-d128-4f6b-b12e-93cbf5be23bd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.070160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.072556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.073148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.073731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.075564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.078439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.078841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.079362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.079651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.081030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] PATH_INFO: `/v3/projects/7dcb543325c04614a6879797e81c89f7/users/090ce3d19b6e4e729ea07c14319db27a/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.101395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 804/1595] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.108822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.112025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:00.113237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:00.121421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dcb543325c04614a6879797e81c89f7, user_id=090ce3d19b6e4e729ea07c14319db27a, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.125365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:39:00.125365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:39:00.125365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:39:00.139342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037ce1c-f201-4e26-8fcc-100758aba816 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.182192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 792/1596] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:00 2022] PUT /identity/v3/projects/7dcb543325c04614a6879797e81c89f7/users/090ce3d19b6e4e729ea07c14319db27a/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:00.185515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.186648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.187557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.273371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] MFA Rules not processed for user `2b4ce342ddf14eef97b7c112a2bdd387`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:00.275048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:00.293329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788fffb-87d1-4fb0-896d-327ac17983a8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.396787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 793/1597] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:00.400666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.401354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.401687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.401799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.402039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.489363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6f66-41c9-4de5-8c1a-706c95fb5164 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.490283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.492335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.495604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.496296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.496584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.496837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.497642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.500087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.506001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 805/1598] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:00.507054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.512899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 794/1599] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.513474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.520967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56facc25-13a6-4cff-9a7c-4f2ea52b18bc service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.525443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef870-bbfe-4fb5-bcb7-98210f29874c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.530438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.531133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.531437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.531689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.532452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.534108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.538040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.538773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.539793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.540147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.541068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.544250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2789-0d46-451c-9b88-691182087260 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.550955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 795/1600] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.550955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.557735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 806/1601] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.559463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.559871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.560300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.569460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5817cf7-6ee1-44d3-bbf3-54b703a3790d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.575167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.576687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.576971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.577246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.578313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.579992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.595248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] MFA Rules not processed for user `090ce3d19b6e4e729ea07c14319db27a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:00.596966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:00.598228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 796/1602] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.599124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.603064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:00.620582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499e84f-7283-4735-aa44-0476a8806604 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.629452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.630632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.630632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.630857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.631548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.634794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.681403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 797/1603] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:00 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:00.682794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.714655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 807/1604] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:00.715781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.738725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45211003-8c06-4a8f-a8c3-ce83fd662844 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.743873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:00.785819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83060-6804-428b-b028-5ee415741591 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.792478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.793179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.793459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.793844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.794667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.796348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.809745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 798/1605] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.817063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.828202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5d663-97e8-4b74-9f78-419537e13893 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.838053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.838949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.839251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.839520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.841147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.842965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.844785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.845982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.846321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.847044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] PATH_INFO: `/v3/projects/92697118684e4e219bd5fa9cffbcb74b/users/768e116fb06849569cd5e55641597d7e/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.860459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 799/1606] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:00.861597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.868603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92697118684e4e219bd5fa9cffbcb74b, user_id=768e116fb06849569cd5e55641597d7e, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.871478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.887960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c04f-329d-442d-a2cb-6e4f7b257e0f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.890227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 808/1607] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:00 2022] PUT /identity/v3/projects/92697118684e4e219bd5fa9cffbcb74b/users/768e116fb06849569cd5e55641597d7e/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:00.891411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:00.967375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.968646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.969035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.969416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] PATH_INFO: `/v3/projects/dbd9c6bf5b0c491299815d545c1a594e/users/8bd1061f505d4249ae14c7cbf68a950a/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.977040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e835-8e71-4eb3-af9c-7f2c51258756 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:00.984759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:00.984759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:00.984759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:00.984759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:00.985530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.987618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87210e88-0d87-4595-b90d-d58b57411b4f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:00.990244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd9c6bf5b0c491299815d545c1a594e, user_id=8bd1061f505d4249ae14c7cbf68a950a, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:00.993949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.016548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 809/1609] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:00 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:01.022741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 800/1609] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:00 2022] PUT /identity/v3/projects/dbd9c6bf5b0c491299815d545c1a594e/users/8bd1061f505d4249ae14c7cbf68a950a/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:01.022741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.023034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def1a1-6a79-4156-914f-a17a7879f5ae None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.023034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def1a1-6a79-4156-914f-a17a7879f5ae None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.023034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def1a1-6a79-4156-914f-a17a7879f5ae None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.027875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 810/1610] 10.222.0.77 () {54 vars in 962 bytes} [Mon Jul 18 16:39:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:01.028850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2def1a1-6a79-4156-914f-a17a7879f5ae None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.063296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:01.146139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8516-2001-42e2-a3b1-21c45b8ed624 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:01.154187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2def1a1-6a79-4156-914f-a17a7879f5ae None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:01.159262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:01.159607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:01.160264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.160578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.160864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] PATH_INFO: `/v3/projects/49d4c19f14be46208297e4e3fb854bb0/users/4a0fa5ed18f2499488e01dd95a65194d/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.163711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.166577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.167578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] PATH_INFO: `/v3/projects/17bce4e79492457db960343dd5219527/users/56da63c91754480e909d5da322687243/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.185896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d4c19f14be46208297e4e3fb854bb0, user_id=4a0fa5ed18f2499488e01dd95a65194d, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:01.189585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1803c0-91a7-4829-a768-d2188f53cd41 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.211088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bce4e79492457db960343dd5219527, user_id=56da63c91754480e909d5da322687243, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:01.215185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3e045-cc43-4480-9232-c80697cf9f8e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.224650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 811/1611] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:01 2022] PUT /identity/v3/projects/49d4c19f14be46208297e4e3fb854bb0/users/4a0fa5ed18f2499488e01dd95a65194d/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:01.227179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.227179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.227179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.252844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 801/1612] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:01 2022] PUT /identity/v3/projects/17bce4e79492457db960343dd5219527/users/56da63c91754480e909d5da322687243/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:01.258883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22bf9-405c-4960-8341-8000acde46eb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.259713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22bf9-405c-4960-8341-8000acde46eb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.260330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f22bf9-405c-4960-8341-8000acde46eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.277082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] MFA Rules not processed for user `768e116fb06849569cd5e55641597d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.280980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.301586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f7f22bf9-405c-4960-8341-8000acde46eb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.309422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f7f22bf9-405c-4960-8341-8000acde46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.358719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:01.409041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 812/1613] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.411015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.438395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 802/1614] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.439960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.440839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.441423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.585047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.585047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.585548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33709ff-0065-47b3-8b60-35432fba0481 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:01.585548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:01.585548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.585548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.586075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.586075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:01.592784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051b279-7557-4575-b7f6-a0bc5a05b686 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.610913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 813/1615] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:01.612310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877fa0-8ab5-4a77-9ddf-e5a9ad856924 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.612610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877fa0-8ab5-4a77-9ddf-e5a9ad856924 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.612953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877fa0-8ab5-4a77-9ddf-e5a9ad856924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.663948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-21877fa0-8ab5-4a77-9ddf-e5a9ad856924 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.666115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-21877fa0-8ab5-4a77-9ddf-e5a9ad856924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.681075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 814/1616] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.688691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.689955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.690498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.738091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] MFA Rules not processed for user `56da63c91754480e909d5da322687243`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.740723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.761313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:01.781312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 803/1617] 10.222.0.77 () {58 vars in 1059 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.782338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.902327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e0c5b-00a0-4a95-97f9-817f990ec66b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:01.909643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:01.912410 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.912833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.913187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.924951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 815/1618] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.927157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.935728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:01.939154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.968313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 804/1619] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:01.973207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.005723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8898e-0fc7-461b-ac61-90fd0afa25a9 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.021328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.023333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.023333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.023333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] PATH_INFO: `/v3/projects/d04aa3283f0d47079805f78a11005321/users/272537f32b4f4cf2b4b61f314b580f24/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.068292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04aa3283f0d47079805f78a11005321, user_id=272537f32b4f4cf2b4b61f314b580f24, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.089051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3068c9f-a852-4cef-bfd7-0367d33f5d13 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.094810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.099435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.100262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.100612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.100968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.126263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.129913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.157434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 816/1620] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:02.158497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.169109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 805/1621] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:01 2022] PUT /identity/v3/projects/d04aa3283f0d47079805f78a11005321/users/272537f32b4f4cf2b4b61f314b580f24/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:02.170874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.193753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257b11-0ca3-409c-bab2-0eed416cb14f service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.243349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:02.343806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.344885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.345398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.345624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23edd59-373b-43e4-aa85-0c5b31d85691 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.346152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.355078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.355764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.356054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.356335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.461888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:02.496655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.499010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.513140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 817/1622] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:02.514304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.527162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e47b6-f1d6-492c-9b70-4927910052e9 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.533981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.534995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.535368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.535608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.535744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.536546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.537542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.539058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.573810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 806/1623] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:02.577451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.600677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 818/1624] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:02 2022] POST /identity/v3/projects => generated 405 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:02.601714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.611190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce72ed-6f45-407e-9b23-9d341248b28b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.788206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.788906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.789227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.789512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.790286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.792124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.814691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6233ff-9962-4f8a-98ed-309e434a81dd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.822362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.823157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.823634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.823927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.824660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.828579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.833696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 807/1625] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:02.834704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.863530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 819/1626] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:02 2022] POST /identity/v3/users => generated 309 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:02.865371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.866008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da107d8-403c-43b2-affa-87ec8122b842 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.879603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.879905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.880086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.880257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] PATH_INFO: `/v3/projects/d04aa3283f0d47079805f78a11005321/users/272537f32b4f4cf2b4b61f314b580f24/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.882185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f57afa-22ae-4801-9a50-21df88225005 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:02.888271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04aa3283f0d47079805f78a11005321, user_id=272537f32b4f4cf2b4b61f314b580f24, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.889311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:02.890382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.890849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.891203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:02.891364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.892123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:02.894577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24922f69-894b-46bf-ac8a-67bb64f2b82b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:02.918358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 808/1627] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:02 2022] PUT /identity/v3/projects/d04aa3283f0d47079805f78a11005321/users/272537f32b4f4cf2b4b61f314b580f24/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:02.920245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 820/1628] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:02.928776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:02.931813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:02.932569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:02.933367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.074096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] MFA Rules not processed for user `272537f32b4f4cf2b4b61f314b580f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:03.083725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:03.098187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:03.108799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989a65-1dd2-44ef-9c4c-80b91db42637 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:03.127882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:03.129252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.129540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.129817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] PATH_INFO: `/v3/projects/03decb0ae21b4e03b93f95965017ed35/users/eb21ad976b814483869d20d49baee9da/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.166031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03decb0ae21b4e03b93f95965017ed35, user_id=eb21ad976b814483869d20d49baee9da, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:03.172260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ff2cb-61f9-4e24-99f9-7d7555e7afd6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:03.208512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 809/1629] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:02 2022] PUT /identity/v3/projects/03decb0ae21b4e03b93f95965017ed35/users/eb21ad976b814483869d20d49baee9da/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:03.212814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdab062-6501-4f87-9568-1f89e69a7e40 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.213153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdab062-6501-4f87-9568-1f89e69a7e40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.213417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdab062-6501-4f87-9568-1f89e69a7e40 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.217257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 810/1630] 10.222.0.77 () {54 vars in 912 bytes} [Mon Jul 18 16:39:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:39:03.218283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdab062-6501-4f87-9568-1f89e69a7e40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:03.364825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdab062-6501-4f87-9568-1f89e69a7e40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:03.393701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:03.394950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.396348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.398797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.400820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:03.405948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92234d95-080e-49b9-b61c-fde7bb142e80 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:03.431675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 811/1631] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:03.433810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74022e-1413-49a7-9b2a-d29b532938e7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.434293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74022e-1413-49a7-9b2a-d29b532938e7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.434741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74022e-1413-49a7-9b2a-d29b532938e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.446040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 821/1632] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:03.447543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:03.479338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4044e-fdbf-4422-af08-0d25bc1d4201 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:03.481188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0b74022e-1413-49a7-9b2a-d29b532938e7 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:03.482436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0b74022e-1413-49a7-9b2a-d29b532938e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:03.489479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:03.490594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.490877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.491151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] PATH_INFO: `/v3/projects/03decb0ae21b4e03b93f95965017ed35/users/eb21ad976b814483869d20d49baee9da/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.497591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03decb0ae21b4e03b93f95965017ed35, user_id=eb21ad976b814483869d20d49baee9da, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:03.502396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:03.537249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 822/1633] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:03 2022] PUT /identity/v3/projects/03decb0ae21b4e03b93f95965017ed35/users/eb21ad976b814483869d20d49baee9da/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:03.538348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:03.583159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:03.662552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 812/1634] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:03.663975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.664289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.664534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.683514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a2274-808c-4cab-965b-34b53c4023b5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:03.698554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:03.699266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.699638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.699792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.747279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] MFA Rules not processed for user `eb21ad976b814483869d20d49baee9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:03.748522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:04.030122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:04.031646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:04.040576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:04.072758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 823/1635] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:04.074848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:04.090287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bb7f7-b1e4-4e19-89e1-2731282beafc service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:04.099617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 813/1636] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:04.104180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:04.104868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:04.105150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:04.105479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:04.285230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:04.287554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:04.332341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:04.344987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081ded9-a22f-4e9d-a677-2985f2d5c89a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:04.349571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 824/1637] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:04.367216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:04.369879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:04.375453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:04.376567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:04.418797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:04.421842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:04.473540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 814/1638] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:04 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:05.039041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:05.087571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1dd30-ec00-4612-a1b8-f8c59e5be7e2 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:05.114175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:05.123149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:05.123919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:05.125707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:05.324867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:05.374582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:05.388382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436769ce-f4b3-4593-bc68-17482461793c service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:05.423943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 825/1639] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:06.051395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:06.221683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483ae2d-e574-44fb-a2bb-5f0f7f41a619 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:06.233855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:06.234810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.242049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.253383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] PATH_INFO: `/v3/users/56da63c91754480e909d5da322687243` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.256240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=56da63c91754480e909d5da322687243)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:06.266952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:39:06.266952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jul 18 16:39:06.266952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: Jul 18 16:39:06.268245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:06.511789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56da63c91754480e909d5da322687243'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:06.535083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f83b4-2115-47e2-9551-7e16faeb14b0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.535658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f83b4-2115-47e2-9551-7e16faeb14b0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.535967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f83b4-2115-47e2-9551-7e16faeb14b0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.548311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 826/1640] 10.222.0.77 () {54 vars in 962 bytes} [Mon Jul 18 16:39:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:06.550442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56da63c91754480e909d5da322687243'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:06.564458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fbb7-8e20-429c-92b5-0d786540e55d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.564961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fbb7-8e20-429c-92b5-0d786540e55d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.565461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fbb7-8e20-429c-92b5-0d786540e55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.640460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f4d0fbb7-8e20-429c-92b5-0d786540e55d None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:06.642467 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f4d0fbb7-8e20-429c-92b5-0d786540e55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:06.658897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56da63c91754480e909d5da322687243'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:06.659853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56da63c91754480e909d5da322687243'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:06.661831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-25d3bdad-7164-4312-b1c1-26197dfbf3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56da63c91754480e909d5da322687243'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:06.699356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 827/1641] 10.222.0.77 () {58 vars in 1059 bytes} [Mon Jul 18 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:06.713063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f5232-2cf8-4a2c-8e01-e0c2e6d76aa8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.713541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f5232-2cf8-4a2c-8e01-e0c2e6d76aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.714006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f5232-2cf8-4a2c-8e01-e0c2e6d76aa8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.718305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 828/1642] 10.222.0.77 () {54 vars in 912 bytes} [Mon Jul 18 16:39:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:39:06.723258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.723716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.724113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.725190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 815/1643] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:06 2022] DELETE /identity/v3/users/56da63c91754480e909d5da322687243 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:06.730996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.731378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.731692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.765273 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:06.766720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:06.788691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:06.795003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:06.953532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 829/1644] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:06.963925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:06.976582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:06.978401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:07.011638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 816/1645] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:07.116345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2f837-45bb-48ed-8da5-4cb81797b298 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:07.129357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:07.130240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:07.130674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:07.131071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:07.155312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:07.157278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:07.181910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 830/1646] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:07.310923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:07.314512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:07.596876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faa7ce-77fd-4ac0-8ab6-11720782b766 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:07.614884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:07.615648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:07.615935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:07.621153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:07.690603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:07.699589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bd5d5-dfd8-42cd-98ad-86b6c5b72cda service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:07.779212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 817/1647] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:09.681776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:09.689379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:09.770355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fe50e-940c-42a4-b82f-c4d9e22d3025 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:09.788745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:09.791887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:09.792348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:09.792830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] PATH_INFO: `/v3/users/090ce3d19b6e4e729ea07c14319db27a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:09.812402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] RBAC: Authorizing `identity:delete_user(user_id=090ce3d19b6e4e729ea07c14319db27a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:09.817979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:10.015981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090ce3d19b6e4e729ea07c14319db27a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:10.040781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090ce3d19b6e4e729ea07c14319db27a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:10.086954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090ce3d19b6e4e729ea07c14319db27a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:10.094174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:10.121780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090ce3d19b6e4e729ea07c14319db27a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:10.123551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090ce3d19b6e4e729ea07c14319db27a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:10.181269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 831/1648] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:09 2022] DELETE /identity/v3/users/090ce3d19b6e4e729ea07c14319db27a => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:10.183731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:10.184461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:10.184461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:10.227088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:10.228753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:10.453793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 818/1649] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:10.540420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:10.542241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:10.755186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7190-4d8a-487d-a31d-78867672863e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:10.764521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:10.765478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:10.766765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:10.767037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:10.803871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:10.814595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a72c0-301d-42cd-a20c-b8de2e8d7f47 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:10.819430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:10.822363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:10.844446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 832/1650] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:10.849050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13411-4b7e-4a5c-b83d-2ce71b67cc2f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:10.888231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:10.890472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:10.891495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:10.902897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] PATH_INFO: `/v3/users/2b4ce342ddf14eef97b7c112a2bdd387` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:10.916245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4ce342ddf14eef97b7c112a2bdd387)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:10.922678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:11.062468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4ce342ddf14eef97b7c112a2bdd387'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:11.084040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4ce342ddf14eef97b7c112a2bdd387'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:11.115780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4ce342ddf14eef97b7c112a2bdd387'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:11.118918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4ce342ddf14eef97b7c112a2bdd387'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:11.119755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4ce342ddf14eef97b7c112a2bdd387'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:11.151104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 819/1651] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:10 2022] DELETE /identity/v3/users/2b4ce342ddf14eef97b7c112a2bdd387 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:11.153101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:11.153393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:11.153678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:11.188935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:11.192569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:11.360922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 833/1652] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:11.438365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:11.441492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:11.477219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:11.735327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:11.736949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:11.770871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c91bd-2a4b-4bcb-b6e7-52cf7938546a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:11.782392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ec53-6f48-4847-98ea-9481fdd0d891 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:11.800889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:11.801796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:11.802355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:11.802817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:11.822456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:11.833758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:11.834246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:11.834767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] PATH_INFO: `/v3/projects/17bce4e79492457db960343dd5219527` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:11.843507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:11.843752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bce4e79492457db960343dd5219527)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:11.845016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:11.845582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:39:11.845582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jul 18 16:39:11.845582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: Jul 18 16:39:11.855104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:11.903476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 820/1653] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:12.132077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bce4e79492457db960343dd5219527', 'request_id': 'req-b8de1bee-71dc-4596-911a-73c95f7f15ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.194168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bce4e79492457db960343dd5219527', 'request_id': 'req-b8de1bee-71dc-4596-911a-73c95f7f15ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.216267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bce4e79492457db960343dd5219527', 'request_id': 'req-b8de1bee-71dc-4596-911a-73c95f7f15ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.245746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] The token cache is being invalidate because project 17bce4e79492457db960343dd5219527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:12.246383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.247684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.254791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 834/1654] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:11 2022] DELETE /identity/v3/projects/17bce4e79492457db960343dd5219527 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:12.258689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:12.275733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703733b-d848-4773-a68f-56638dd07b6a service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:12.419023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:12.420235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:12.420689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:12.421309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] PATH_INFO: `/v3/users/eb21ad976b814483869d20d49baee9da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:12.424992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb21ad976b814483869d20d49baee9da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:12.427317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:12.535232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb21ad976b814483869d20d49baee9da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.558454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb21ad976b814483869d20d49baee9da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.591480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb21ad976b814483869d20d49baee9da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.606983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb21ad976b814483869d20d49baee9da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.608333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b158bc98-b3af-4b63-9524-2552fac98e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb21ad976b814483869d20d49baee9da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:12.632162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 821/1655] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:12 2022] DELETE /identity/v3/users/eb21ad976b814483869d20d49baee9da => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:14.219999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:14.223984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:14.250722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1bee-71dc-4596-911a-73c95f7f15ce admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:14.305310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:14.419272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:14.420533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:14.421029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:14.421468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] PATH_INFO: `/v3/projects/8ff24b729d5747ca8bc7c6bb5e8293c1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:14.423283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff24b729d5747ca8bc7c6bb5e8293c1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:14.425461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:14.549438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff24b729d5747ca8bc7c6bb5e8293c1', 'request_id': 'req-10fbbbf4-5970-441a-86de-566657c78aae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:14.571307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff24b729d5747ca8bc7c6bb5e8293c1', 'request_id': 'req-10fbbbf4-5970-441a-86de-566657c78aae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:14.599930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff24b729d5747ca8bc7c6bb5e8293c1', 'request_id': 'req-10fbbbf4-5970-441a-86de-566657c78aae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:14.641908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] The token cache is being invalidate because project 8ff24b729d5747ca8bc7c6bb5e8293c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:14.655236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:14.674123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:14.685022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 835/1656] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:14 2022] DELETE /identity/v3/projects/8ff24b729d5747ca8bc7c6bb5e8293c1 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:14.691726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:14.692529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:14.692529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:14.700516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:14.707806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:14.792795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] MFA Rules not processed for user `4909d4aef302404290adcd3268e53c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:14.794670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:14.869603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:15.010311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbbbf4-5970-441a-86de-566657c78aae admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:15.033657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:15.036423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.036938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.037444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] PATH_INFO: `/v3/users/272537f32b4f4cf2b4b61f314b580f24` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.049122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=272537f32b4f4cf2b4b61f314b580f24)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:15.069005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:15.070905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 822/1657] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:15.072728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:15.075318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.135629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad265f-7958-42e0-a3d3-6227cb70aa70 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:15.160457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:15.162805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.163117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.167946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] PATH_INFO: `/v3/projects/7dcb543325c04614a6879797e81c89f7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.172282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dcb543325c04614a6879797e81c89f7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:15.182870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:15.307342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272537f32b4f4cf2b4b61f314b580f24'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.326046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:15.372736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272537f32b4f4cf2b4b61f314b580f24'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.436130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272537f32b4f4cf2b4b61f314b580f24'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.450469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dcb543325c04614a6879797e81c89f7', 'request_id': 'req-922f6890-e7fe-4c5a-b467-0ee8d3903527'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.472223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272537f32b4f4cf2b4b61f314b580f24'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.479515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272537f32b4f4cf2b4b61f314b580f24'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.493483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dcb543325c04614a6879797e81c89f7', 'request_id': 'req-922f6890-e7fe-4c5a-b467-0ee8d3903527'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.503801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dcb543325c04614a6879797e81c89f7', 'request_id': 'req-922f6890-e7fe-4c5a-b467-0ee8d3903527'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.516957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 836/1658] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:14 2022] DELETE /identity/v3/users/272537f32b4f4cf2b4b61f314b580f24 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:15.521040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:15.530754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.545188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] The token cache is being invalidate because project 7dcb543325c04614a6879797e81c89f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:15.546359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.552467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:15.559674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 823/1659] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:15 2022] DELETE /identity/v3/projects/7dcb543325c04614a6879797e81c89f7 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:15.567737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:15.571336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.866212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910da4dc-b292-4dfc-aabd-5b25210b1996 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:15.868485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6890-e7fe-4c5a-b467-0ee8d3903527 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:15.875842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:15.878043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.878886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.879768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] PATH_INFO: `/v3/projects/03decb0ae21b4e03b93f95965017ed35` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.883293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03decb0ae21b4e03b93f95965017ed35)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:15.888971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:15.973274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:16.012731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:16.015754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.016431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.016905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.021273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:16.069002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03decb0ae21b4e03b93f95965017ed35', 'request_id': 'req-4e81015e-8722-4456-aae3-01ac8336f5d1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:16.103388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03decb0ae21b4e03b93f95965017ed35', 'request_id': 'req-4e81015e-8722-4456-aae3-01ac8336f5d1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:16.133415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03decb0ae21b4e03b93f95965017ed35', 'request_id': 'req-4e81015e-8722-4456-aae3-01ac8336f5d1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:16.143218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:16.146092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac7965-9179-4b19-8cfc-76b42261d5a5 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:16.163134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 824/1660] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:16.167417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aa243-89f3-442a-8c5a-8b3d484b797b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.167913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aa243-89f3-442a-8c5a-8b3d484b797b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.169467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aa243-89f3-442a-8c5a-8b3d484b797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.200495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] The token cache is being invalidate because project 03decb0ae21b4e03b93f95965017ed35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:16.207187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:16.209702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e81015e-8722-4456-aae3-01ac8336f5d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:16.213753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-097aa243-89f3-442a-8c5a-8b3d484b797b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:16.216369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-097aa243-89f3-442a-8c5a-8b3d484b797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:16.217299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 837/1661] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:15 2022] DELETE /identity/v3/projects/03decb0ae21b4e03b93f95965017ed35 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:16.225564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.226739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.227554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.307163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:16.311642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:16.345346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 825/1662] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:16.354092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b9d9-757d-4f9d-bb21-d5642d1278e1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.354829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b9d9-757d-4f9d-bb21-d5642d1278e1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.355317 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b9d9-757d-4f9d-bb21-d5642d1278e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.458178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cdc6b9d9-757d-4f9d-bb21-d5642d1278e1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:16.470759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cdc6b9d9-757d-4f9d-bb21-d5642d1278e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:16.494938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:16.522402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 838/1663] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:16.529432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 826/1664] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:16.531553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:16.537866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.538183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.538515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.561740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b91b0c-abe8-4c03-96ef-bbacdb02a2f0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:16.575001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:16.575944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.576244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.576527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.578796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:16.579172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:16.582519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:16.594146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:16.595547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:16.622640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 839/1665] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:16.628614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:16.638896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 827/1666] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:16.648423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:16.658217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f275-e3f7-4bd8-a16e-16d68874f5aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:16.663099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72f99-5b88-44b0-a3fa-8fa78181c803 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:16.671914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:16.672631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.672921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.673205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.673939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:16.676000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:16.680492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:16.683627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:16.685083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:16.686002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:16.689538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:16.692595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:16.705098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:16.785175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 828/1667] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:16.793760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:16.802444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 840/1668] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/projects => generated 355 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:16.809053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:16.878754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec251ee-fdf0-4621-98c0-4050063a353e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.023914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2150db-6d8e-4fb1-be5f-35b391dc7db3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.044845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.045582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.045871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.046489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.047279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.050003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.115171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 841/1669] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/users => generated 278 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:17.119493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.146238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5f9a-c890-4443-bba5-ad025240c23a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.162604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.163369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.164355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.164642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.165419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.170263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.203013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 842/1670] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:17.204346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.208573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.237530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:17.257829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ff2a8-582e-458a-9e9b-5affbd4ca58e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.278623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.279795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.281360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.281549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.281712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.281913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] PATH_INFO: `/v3/projects/d04aa3283f0d47079805f78a11005321` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.284309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.284906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.286248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04aa3283f0d47079805f78a11005321)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.288329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.290681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.301864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.387064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 829/1671] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:16 2022] POST /identity/v3/projects => generated 403 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:17.390166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.428767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04aa3283f0d47079805f78a11005321', 'request_id': 'req-145ee980-bb28-43ba-bfae-32ea4999f9b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:17.448949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04aa3283f0d47079805f78a11005321', 'request_id': 'req-145ee980-bb28-43ba-bfae-32ea4999f9b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:17.471475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04aa3283f0d47079805f78a11005321', 'request_id': 'req-145ee980-bb28-43ba-bfae-32ea4999f9b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:17.501000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] The token cache is being invalidate because project d04aa3283f0d47079805f78a11005321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:17.501193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:17.501940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:17.503961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 843/1672] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:17 2022] DELETE /identity/v3/projects/d04aa3283f0d47079805f78a11005321 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:17.505519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.508180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.515533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eacc6-fbca-4056-8adc-d093375c6342 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.568354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:17.614982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad9154-503b-4055-8554-852229660982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.617399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9154-503b-4055-8554-852229660982 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.617960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9154-503b-4055-8554-852229660982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.618620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9154-503b-4055-8554-852229660982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.620593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad9154-503b-4055-8554-852229660982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.622821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad9154-503b-4055-8554-852229660982 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.637832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 830/1673] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:17.640237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.641124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.641613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.686534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ee980-bb28-43ba-bfae-32ea4999f9b0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.703952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.704666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.704944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.705226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.705956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.708329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81422-99d1-4eff-bb01-58ce6383c433 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.714737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] MFA Rules not processed for user `ab3be59ea4c74da3b4e19273102bccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:17.716048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.761329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 844/1674] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:17 2022] POST /identity/v3/users => generated 302 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:17.768133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.768572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.769964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.831816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:17.836082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.851348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 831/1675] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:17.852297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.862673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.865591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 845/1676] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:17.868205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:17.891289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85849ee1-6bf6-42d6-9a4e-40acecb1c294 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.896235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb445a-75c2-4b5b-9484-5f0686baa228 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:17.906984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.911910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.912700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.913193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] PATH_INFO: `/v3/projects/a03e8aab03ca435b9055494dd1bea015/users/3e94fdb746514b5db97343af949426e3/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.930039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:17.933104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.934134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.934905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.945998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.947156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:17.951256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114eaa3-c952-4b0a-b5fc-a8d2f8c0e121 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:17.964084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03e8aab03ca435b9055494dd1bea015, user_id=3e94fdb746514b5db97343af949426e3, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:17.976241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b117bd7-90fa-4e3f-b725-dc339524e68c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:18.010720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 846/1677] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:18.014778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.021203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.022191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 832/1678] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:17 2022] PUT /identity/v3/projects/a03e8aab03ca435b9055494dd1bea015/users/3e94fdb746514b5db97343af949426e3/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:18.022965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.044071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.044518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.044807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.071642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] MFA Rules not processed for user `ab3be59ea4c74da3b4e19273102bccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:18.073378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.092177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:18.096609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.212537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 833/1679] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:18.213856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.220529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.256157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02022ed8-dd6e-4a21-b3f1-1342caf30eb4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.256600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 847/1680] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:18.259049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.289278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902aad8b-e971-477a-a873-517fc09264b3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.296792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:18.301465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:18.301465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.301465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.301465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.302357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:18.303642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.304682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.304979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] PATH_INFO: `/v3/users/768e116fb06849569cd5e55641597d7e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.308560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=768e116fb06849569cd5e55641597d7e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:18.309674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:18.328543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad9018-1c2f-4d80-9add-61388bf2dfe0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:18.351964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 848/1681] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:18.353865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.353865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.354206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.406083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] MFA Rules not processed for user `3e94fdb746514b5db97343af949426e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:18.407668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.450963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768e116fb06849569cd5e55641597d7e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:18.474329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768e116fb06849569cd5e55641597d7e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:18.519812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768e116fb06849569cd5e55641597d7e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:18.526279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768e116fb06849569cd5e55641597d7e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:18.528810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768e116fb06849569cd5e55641597d7e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:18.565700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 834/1682] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:18 2022] DELETE /identity/v3/users/768e116fb06849569cd5e55641597d7e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:18.565700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.567792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.583988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 849/1683] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:18.586593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.590130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.652548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:18.767666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91e73-f644-4286-9cda-120f6a76ef93 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.778315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:18.779292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.779746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.780970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.782360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:18.782912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:18.785051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:18.797263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:18.813440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 835/1684] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:18.815857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.823628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.861068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434843f-b50f-4330-8bc6-7697f395c1d0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.861388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017939-ed6a-4fa6-ac87-4eab6ce6cd4e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.875924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:18.880780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.881902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.883608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] PATH_INFO: `/v3/projects/68e857db505b4759ae2afc9661c2f2b0/users/7ba87da3287d41dc9dff40e02c851030/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.975136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e857db505b4759ae2afc9661c2f2b0, user_id=7ba87da3287d41dc9dff40e02c851030, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:18.978204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0305d01-b498-4b85-a8ec-3699c8f6760d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:19.011067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 836/1685] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:18 2022] PUT /identity/v3/projects/68e857db505b4759ae2afc9661c2f2b0/users/7ba87da3287d41dc9dff40e02c851030/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:19.017215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:19.017685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:19.021388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:19.046694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:19.047682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:19.048146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:19.048571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:19.061785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:19.077864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:19.082153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:19.345711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 837/1686] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:19.346743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:19.351482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:19.363182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:19.367162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:19.388693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41652c73-b9b3-445c-9bf6-d7c86050f01d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:19.393872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:19.394604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:19.394893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:19.395173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:19.395937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:19.397998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b947edd-5aa6-455f-9319-ebf2d424983b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:19.419610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 850/1687] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:19.421104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:19.465840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0ceb-bbaa-48c5-9a8e-23e3382fc6dd service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:19.473376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 838/1688] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:19 2022] POST /identity/v3/projects => generated 357 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:19.475036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:19.475495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:19.478264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:19.524583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:19.526087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:19.687903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 839/1689] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:19.688819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:19.693994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:19.849621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:19.899537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:19.900624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:19.901115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:19.901608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:19.902959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:19.913771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.044362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 851/1690] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:19 2022] POST /identity/v3/projects => generated 387 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:20.045770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.106332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79240ad8-032f-490a-abbb-17dd9730217a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.134991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:20.388288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e99dac-07d2-4947-a0c9-3121a832d8a3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.404490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.405641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.406146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.410377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.411859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.419279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.439003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.440094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.440565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.441010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.482183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 852/1691] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:20 2022] POST /identity/v3/users => generated 285 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:20.483675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.511517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cbd0c-3867-4a26-876a-ce4c05dc2204 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.526334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.527447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.527957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.528443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.532063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.534989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.550892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.552794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.579941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 853/1692] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:20.581237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.613911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716604ab-2533-4f06-8067-da98dbfa4180 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.623533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 840/1693] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 935 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:20.629312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.632905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.633792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.634188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.634605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.635455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.637693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.693266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 854/1694] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:20 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:20.694247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20cc58-c0d0-4d39-a70e-303426ab3506 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.699796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.720616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ef6c-c9f3-4616-a18b-80818c987dba admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.729054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.730189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.730719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.731202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.734748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.735598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.735820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.736132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.736503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.737395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.738767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.744466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.788082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 855/1695] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:39:20.789809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.807212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 841/1696] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:20.808938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.818659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a111f9-fd51-42ee-99d5-c610b1c92075 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.834451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.835927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.836714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.837135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.837995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.841289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.854776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a65f50-cc7a-4b49-b0fa-a21695232d99 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.887673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 856/1697] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:20.888733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.888733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.888733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.888733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] PATH_INFO: `/v3/projects/1731d3615dc54aa5a9513f8daf723d7c/users/6a57e0d0d2a64f07b2879f8f2b04867c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.889589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.911518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874f855-d8b1-41c1-9585-d03ed812a8db admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.924571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:20.925737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.926155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.926591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] PATH_INFO: `/v3/projects/18431c63b9884b508d53eaf28859cba3/users/bf563e8d46b74cfc81314adba5a1ea22/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.959776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1731d3615dc54aa5a9513f8daf723d7c, user_id=6a57e0d0d2a64f07b2879f8f2b04867c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.965973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:20.977939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18431c63b9884b508d53eaf28859cba3, user_id=bf563e8d46b74cfc81314adba5a1ea22, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:20.985516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b815f-a48b-4abe-86a0-df1bebf7e453 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:21.026962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 842/1698] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:20 2022] PUT /identity/v3/projects/1731d3615dc54aa5a9513f8daf723d7c/users/6a57e0d0d2a64f07b2879f8f2b04867c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:21.027424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 857/1699] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:20 2022] PUT /identity/v3/projects/18431c63b9884b508d53eaf28859cba3/users/bf563e8d46b74cfc81314adba5a1ea22/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:21.027575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:21.042464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.042953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.043439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.133047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] MFA Rules not processed for user `bf563e8d46b74cfc81314adba5a1ea22`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:21.135188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:21.257867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:21.309595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 858/1700] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:21.315814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:21.353187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad755-973a-4197-b8bd-1fee8e1635db admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:21.476712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b300c-df4f-4e1e-b2c1-04d31cd98b78 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:21.500237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:21.501070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.501571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.501969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.502993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:21.507552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaa2c5-eae7-46a6-9206-3f7d269db2e3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:21.661076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:21.661983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.662545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.663034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.668694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 859/1701] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:21 2022] POST /identity/v3/projects => generated 387 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:39:21.668927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:21.669082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c9a-3ece-46f4-a8f2-e29e73333b6d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.669082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c9a-3ece-46f4-a8f2-e29e73333b6d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.669082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a990c9a-3ece-46f4-a8f2-e29e73333b6d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.678661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6053b-fcdd-4a17-80e2-dbc703897ed8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:21.710399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 860/1702] 10.222.0.77 () {54 vars in 912 bytes} [Mon Jul 18 16:39:21 2022] GET /identity => generated 270 bytes in 46 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:21.716255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a990c9a-3ece-46f4-a8f2-e29e73333b6d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:21.746177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 843/1703] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 720 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:21.746177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.746177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.746177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.892615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:21.922731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:21.924879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:22.031813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a990c9a-3ece-46f4-a8f2-e29e73333b6d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:22.049577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:22.065607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:22.070230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:22.073176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:22.077861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:22.087593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:22.228205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 861/1704] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:21 2022] POST /identity/v3/users => generated 300 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:22.232097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:22.295084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79566b9-2327-4839-be6f-cee53a1007c6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:22.316094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 844/1705] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:39:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:22.317786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:22.323933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:22.326941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:22.330955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:22.331359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] PATH_INFO: `/v3/projects/1731d3615dc54aa5a9513f8daf723d7c/users/6a57e0d0d2a64f07b2879f8f2b04867c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:22.340133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1731d3615dc54aa5a9513f8daf723d7c, user_id=6a57e0d0d2a64f07b2879f8f2b04867c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:22.352590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa44c94-7338-4992-8627-9ddc620467e4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:22.402201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5508-720d-4ad8-a694-3f578a367530 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:22.408699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 862/1706] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:22 2022] PUT /identity/v3/projects/1731d3615dc54aa5a9513f8daf723d7c/users/6a57e0d0d2a64f07b2879f8f2b04867c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:22.431631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:22.432998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:22.433481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:22.580937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] MFA Rules not processed for user `6a57e0d0d2a64f07b2879f8f2b04867c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:22.582698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:22.778042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:22.796512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:22.797802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:22.798310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:22.803567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:22.806085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:22.817665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:22.851654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 845/1707] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 535 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:22.852943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:22.994322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 863/1708] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:22 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:22.995818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:23.038043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fed405-9b11-44d9-8360-d7a35c53ddec None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.065215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb548940-df8c-4b87-a2ca-396b73d6551d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.076033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.076371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.076663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.076945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] PATH_INFO: `/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e/users/e5e1df434a554a71a0f3b2f40ff1077d/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.109962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.119439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.122843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.123433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.136261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.140299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.172067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43041cf735a4ac98ce8ac651a4caf6e, user_id=e5e1df434a554a71a0f3b2f40ff1077d, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.180151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.187724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 846/1709] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:23.191974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:23.199158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:23.247236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 864/1710] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:22 2022] PUT /identity/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e/users/e5e1df434a554a71a0f3b2f40ff1077d/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:23.248681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:23.417694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716af3-661f-4b0f-9b70-e2832f6cc942 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.433245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379f35-5415-4e6d-b72b-5f704c80d2d7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.443268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:23.446480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.447208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.448809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.449137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.452340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.454114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.467278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 865/1711] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:23.472651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:23.502838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50874a9d-9d74-44e3-bc09-23320195adaf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.515059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.518171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.518429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.518687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] PATH_INFO: `/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e/users/e5e1df434a554a71a0f3b2f40ff1077d/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.522387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43041cf735a4ac98ce8ac651a4caf6e, user_id=e5e1df434a554a71a0f3b2f40ff1077d, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.532752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee74640-7a85-411c-aa6e-087e86d2c1bf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.582197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.582956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.583240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.583520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.602728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 866/1712] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:23 2022] PUT /identity/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e/users/e5e1df434a554a71a0f3b2f40ff1077d/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:23.621917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.622505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.622957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.671220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] MFA Rules not processed for user `e5e1df434a554a71a0f3b2f40ff1077d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:23.693585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:23.787331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.789695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.827185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 847/1713] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:23 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:24.048770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 867/1714] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:24.058125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:24.076651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94888c-10af-40b7-90a1-f06fc7432bc1 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:24.158994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:24.161954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:24.208199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:24.255918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:24.257962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:24.258737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:24.276887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:24.322122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:24.326879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32fd11-5f03-427b-b9c4-9d93c8441432 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:24.336222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf178fd-bc18-4a17-b644-08c5e2444903 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:24.347022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:24.347863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:24.348430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:24.349681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:24.362090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 848/1715] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:24.412594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:24.413176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:24.413573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:24.518586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:24.520466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:24.548359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 868/1716] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:24 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:24.566850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] MFA Rules not processed for user `8bd1061f505d4249ae14c7cbf68a950a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:24.571302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:24.731202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:24.862142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:24.934124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 849/1717] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:24.952816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:24.957690 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:25.009026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d45-fd25-47d9-859b-e683d2ac5c34 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:25.047787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:25.051739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.052681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.052681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] PATH_INFO: `/v3/users/bf563e8d46b74cfc81314adba5a1ea22` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.059729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf563e8d46b74cfc81314adba5a1ea22)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:25.062123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:25.191057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf563e8d46b74cfc81314adba5a1ea22'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:25.218107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf563e8d46b74cfc81314adba5a1ea22'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:25.248298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:25.255643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf563e8d46b74cfc81314adba5a1ea22'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:25.272158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5046c1-c05b-48bb-84cf-e9f431606f44 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:25.274006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf563e8d46b74cfc81314adba5a1ea22'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:25.274851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-26f43370-05d7-4378-b08e-862c7e85a6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf563e8d46b74cfc81314adba5a1ea22'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:25.287699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 869/1718] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:24 2022] DELETE /identity/v3/users/bf563e8d46b74cfc81314adba5a1ea22 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:25.290787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e1a2-a591-4b09-be4b-c629ce4ef92c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.291211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e1a2-a591-4b09-be4b-c629ce4ef92c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.291611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67e1a2-a591-4b09-be4b-c629ce4ef92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.299733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:25.312306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.316361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.317430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.324206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:25.334741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b67e1a2-a591-4b09-be4b-c629ce4ef92c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:25.343624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b67e1a2-a591-4b09-be4b-c629ce4ef92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:25.389469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:25.392599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1ff9-5723-414d-ade7-98cdd98f67bd service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:25.423068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 850/1719] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:25.561530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 870/1720] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:25.613295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.613811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.614279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.617705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 851/1721] 10.222.0.77 () {54 vars in 961 bytes} [Mon Jul 18 16:39:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:25.631564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44c89-e1ed-4de2-a4e9-eec4b37889a4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.632417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44c89-e1ed-4de2-a4e9-eec4b37889a4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.632726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44c89-e1ed-4de2-a4e9-eec4b37889a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.690204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d6e44c89-e1ed-4de2-a4e9-eec4b37889a4 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:25.693021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d6e44c89-e1ed-4de2-a4e9-eec4b37889a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:25.719979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:25.724571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.024389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 871/1722] 10.222.0.77 () {58 vars in 1058 bytes} [Mon Jul 18 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:26.025934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5c4a-d25f-4308-b9e2-67d282d693bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.026218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5c4a-d25f-4308-b9e2-67d282d693bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.026526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5c4a-d25f-4308-b9e2-67d282d693bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.054905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d57f5c4a-d25f-4308-b9e2-67d282d693bb None None] MFA Rules not processed for user `4909d4aef302404290adcd3268e53c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:26.067259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d57f5c4a-d25f-4308-b9e2-67d282d693bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.083381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb867bd3-6664-488c-a7b5-d5558deb3b6a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:26.103041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:26.104051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.104558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.106048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.158632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:26.165543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:26.184595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 872/1723] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:26.192685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da797-35a5-4d8e-abb1-e24c3719d53c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.193174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da797-35a5-4d8e-abb1-e24c3719d53c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.193589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da797-35a5-4d8e-abb1-e24c3719d53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.230950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 852/1724] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:26.235110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:26.248847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-320da797-35a5-4d8e-abb1-e24c3719d53c None None] MFA Rules not processed for user `ab3be59ea4c74da3b4e19273102bccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:26.254022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-320da797-35a5-4d8e-abb1-e24c3719d53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.299682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2733d14-7493-40db-9486-0dca779eb91a service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:26.336537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:26.337648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.338157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.338670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.339023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:26.347212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.426173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:26.429544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:26.462043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 873/1725] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:26.491566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6e6e-bf56-41dd-90ff-32eaa0cec718 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.491883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6e6e-bf56-41dd-90ff-32eaa0cec718 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.492166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6e6e-bf56-41dd-90ff-32eaa0cec718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.528441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 853/1726] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:26.592107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e7ef6e6e-bf56-41dd-90ff-32eaa0cec718 None None] MFA Rules not processed for user `4a0fa5ed18f2499488e01dd95a65194d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:26.595579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e7ef6e6e-bf56-41dd-90ff-32eaa0cec718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.661749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:26.671974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.806290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 874/1727] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:26.821515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047db691-b966-4bc3-be57-2d6c04dd8209 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:26.828345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a711f-9a46-4a38-a553-8468f8c9ce3c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.828642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a711f-9a46-4a38-a553-8468f8c9ce3c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.828932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a711f-9a46-4a38-a553-8468f8c9ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.853191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:26.862617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.863821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.865938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] PATH_INFO: `/v3/projects/92697118684e4e219bd5fa9cffbcb74b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.867658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-523a711f-9a46-4a38-a553-8468f8c9ce3c None None] MFA Rules not processed for user `4a0fa5ed18f2499488e01dd95a65194d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:26.876087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-523a711f-9a46-4a38-a553-8468f8c9ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.876547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92697118684e4e219bd5fa9cffbcb74b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:26.897009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:26.922767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 875/1728] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:27.103155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:27.104243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:27.104243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:27.168047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] MFA Rules not processed for user `7ba87da3287d41dc9dff40e02c851030`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:27.169358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:27.192599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92697118684e4e219bd5fa9cffbcb74b', 'request_id': 'req-10f8f66f-63ba-4b08-a611-3c804e81e150'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:27.230898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92697118684e4e219bd5fa9cffbcb74b', 'request_id': 'req-10f8f66f-63ba-4b08-a611-3c804e81e150'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:27.253116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92697118684e4e219bd5fa9cffbcb74b', 'request_id': 'req-10f8f66f-63ba-4b08-a611-3c804e81e150'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:27.262292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:27.299655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] The token cache is being invalidate because project 92697118684e4e219bd5fa9cffbcb74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:27.301357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:27.309101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-10f8f66f-63ba-4b08-a611-3c804e81e150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:27.319072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 854/1729] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:26 2022] DELETE /identity/v3/projects/92697118684e4e219bd5fa9cffbcb74b => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:27.324944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d7cab-9b34-449b-90ea-34649d9fd5ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:27.326912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d7cab-9b34-449b-90ea-34649d9fd5ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:27.327253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d7cab-9b34-449b-90ea-34649d9fd5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:27.407020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-301d7cab-9b34-449b-90ea-34649d9fd5ea None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:27.414820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-301d7cab-9b34-449b-90ea-34649d9fd5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:27.467396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 876/1730] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:27.469233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:27.479465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:27.669600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 855/1731] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:27.670249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:27.692957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:27.693216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:27.695334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:27.738093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810a815-043f-4b6c-9b35-72c651b1fd57 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:27.761360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:27.762068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:27.762361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:27.766351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:27.769568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:27.775414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:27.797953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:27.812005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:27.859464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 856/1732] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:27.861506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:27.869913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 877/1733] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:27 2022] POST /identity/v3/projects => generated 405 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:27.874359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:27.890598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e8d27-4178-44c2-902e-5e86d8e7390b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.161626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.162541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.162817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.163148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.167929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:28.168284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:28.173873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:28.182442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19417a89-def3-4470-ac45-479a7420e923 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.216676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 857/1734] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:28.226137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:28.231111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.235047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.238082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.239640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.244689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:28.256559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a9e61-8a0d-4f8c-898e-6fcc9cad3987 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.258177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:28.295085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.296543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.298892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.299523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.305743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:28.311364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:28.357081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 878/1735] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:27 2022] POST /identity/v3/users => generated 309 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:28.364672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:28.413116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8165-65d9-4ed8-8222-f71919eda32a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.431743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 858/1736] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:28 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:28.445841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:28.447028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.447775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.448560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.454790 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.456829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:28.458609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:28.519527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 879/1737] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:28.523708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:28.565665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12725-c898-4630-8c9d-396e18785de0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.740340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:28.758685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af79bc8-a13a-4624-b4b0-fa01de5bec22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:28.792691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.795184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.797226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.797402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.798178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:28.841266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3b175-44aa-4ea7-b7c7-eee90d8ab5b6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:28.963156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:28.974208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.980983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:28.981673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] PATH_INFO: `/v3/projects/396b1d78b1e4487eb57c5238aa2c8565/users/1eb2f46ba1fb498885c671280815d875/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:28.988204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 859/1738] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:28 2022] POST /identity/v3/users => generated 288 bytes in 549 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:28.999090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c297ad-ca96-43df-b78e-374eab97b28a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:28.999602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c297ad-ca96-43df-b78e-374eab97b28a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.000137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c297ad-ca96-43df-b78e-374eab97b28a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.010679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 860/1739] 10.222.0.77 () {54 vars in 912 bytes} [Mon Jul 18 16:39:28 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:29.034934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c297ad-ca96-43df-b78e-374eab97b28a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.086324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] RBAC: Authorizing `identity:create_grant(project_id=396b1d78b1e4487eb57c5238aa2c8565, user_id=1eb2f46ba1fb498885c671280815d875, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.093714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da7e12-e127-425a-affc-75916eadadab admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.098071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c297ad-ca96-43df-b78e-374eab97b28a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.113944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.115771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.118265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.120294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.122238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.126633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.170057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 861/1741] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:29.170225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 880/1741] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:28 2022] PUT /identity/v3/projects/396b1d78b1e4487eb57c5238aa2c8565/users/1eb2f46ba1fb498885c671280815d875/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:29.172174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.173556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.175450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.176414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.200940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2c465-6a14-4b22-bd4e-ba3067497e56 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.223600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:29.226375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:29.355882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:29.389109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.390629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.391129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.391749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.393539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.400526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.420886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 881/1742] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:39:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:29.423094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.432991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 862/1743] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:29.435876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.438713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:29.452929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2abe32-d614-4243-abb5-84f06ec67d2c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.466361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.468514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.469153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.470983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7178f6-48b6-4121-8618-8e4a14690c9e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.471742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.472849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.478145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.490948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.492162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.493858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.494268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] PATH_INFO: `/v3/projects/18431c63b9884b508d53eaf28859cba3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.496072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18431c63b9884b508d53eaf28859cba3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.506260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 882/1744] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:29.512476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.512626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.565095 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e56185-d2f4-4bc7-970a-cf6549a66879 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.603580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.607058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.607352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.608997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] PATH_INFO: `/v3/projects/396b1d78b1e4487eb57c5238aa2c8565/users/1eb2f46ba1fb498885c671280815d875/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:29.619951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396b1d78b1e4487eb57c5238aa2c8565, user_id=1eb2f46ba1fb498885c671280815d875, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:29.622988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:29.670389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 883/1745] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:29 2022] PUT /identity/v3/projects/396b1d78b1e4487eb57c5238aa2c8565/users/1eb2f46ba1fb498885c671280815d875/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:29.671579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.679105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:29.703723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18431c63b9884b508d53eaf28859cba3', 'request_id': 'req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:29.727457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18431c63b9884b508d53eaf28859cba3', 'request_id': 'req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:29.739583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18431c63b9884b508d53eaf28859cba3', 'request_id': 'req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:29.794210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] The token cache is being invalidate because project 18431c63b9884b508d53eaf28859cba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:29.794837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:29.796088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:29.804934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 863/1746] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:29 2022] DELETE /identity/v3/projects/18431c63b9884b508d53eaf28859cba3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:29.804934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:29.808013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:29.946613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:29.950323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf8a6c-d3cf-43c9-8666-e4034cfadb90 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.964031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bdeb-ce1d-4d03-aa5b-fe9ff6866f45 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:29.969781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:29.970794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:29.971074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:29.971357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] PATH_INFO: `/v3/projects/9722e5dc37ca423592891a601f0a0632/users/4a36240c9a3d48c2b17ec0f275073f22/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:30.012273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9722e5dc37ca423592891a601f0a0632, user_id=4a36240c9a3d48c2b17ec0f275073f22, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:30.014664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:30.054067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 864/1747] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:29 2022] PUT /identity/v3/projects/9722e5dc37ca423592891a601f0a0632/users/4a36240c9a3d48c2b17ec0f275073f22/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:30.055453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:30.064375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:30.127654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:30.129501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:30.129937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:30.130326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:30.134248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:30.169824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:30.171575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:30.196081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 884/1748] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:30.198455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:30.203107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:30.240301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50857f28-910b-4178-870b-bebeb7566519 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:30.282812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbe760-2913-448b-aedf-b0630174e0f0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:30.299747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:30.300464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:30.300755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:30.301037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:30.303937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:30.440930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:30.448928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:30.450478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:30.450966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] PATH_INFO: `/v3/users/e5e1df434a554a71a0f3b2f40ff1077d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:30.455990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e1df434a554a71a0f3b2f40ff1077d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:30.462857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:30.480502 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:30.491720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc973b5d-5f75-4792-8205-c45803a93498 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:30.518652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 865/1749] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:30.521812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:30.522904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:30.523460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:30.531085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:30.605746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e1df434a554a71a0f3b2f40ff1077d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:30.618444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] MFA Rules not processed for user `4a36240c9a3d48c2b17ec0f275073f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:30.619743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:30.637132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e1df434a554a71a0f3b2f40ff1077d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:30.698549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e1df434a554a71a0f3b2f40ff1077d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:30.725302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e1df434a554a71a0f3b2f40ff1077d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:30.726202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-739376ad-9014-4320-8a6e-07c83cc4b655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e1df434a554a71a0f3b2f40ff1077d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:30.746923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 885/1750] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:30 2022] DELETE /identity/v3/users/e5e1df434a554a71a0f3b2f40ff1077d => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:30.814573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 866/1751] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:32.182779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85725a-cbcf-43bd-8bea-f8ded46503b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:32.183567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85725a-cbcf-43bd-8bea-f8ded46503b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:32.183567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85725a-cbcf-43bd-8bea-f8ded46503b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:32.336601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-df85725a-cbcf-43bd-8bea-f8ded46503b4 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:32.339761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-df85725a-cbcf-43bd-8bea-f8ded46503b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:32.561240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 886/1752] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:32.965867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:32.969650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:32.978521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:33.048554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:33.049054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:33.049633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:33.123691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] MFA Rules not processed for user `1eb2f46ba1fb498885c671280815d875`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:33.127262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:33.305413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766909f-fdfb-46d6-adef-a6a537bb5d13 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:33.336199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:33.399506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 887/1753] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:33.400453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:33.405149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:33.489913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:33.490655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:33.492076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:33.492418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:33.504466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:33.507927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590731dc-7766-4e52-b26d-f4fe403dfeac service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:33.537499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 867/1754] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:39:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:39:33.599252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:33.599555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:33.599836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:33.652860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] MFA Rules not processed for user `8bd1061f505d4249ae14c7cbf68a950a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:33.654166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:33.748312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e0de8-3fd7-4d6e-a612-c6c580782fa2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:33.772991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:33.779169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:33.779540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:33.779831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:33.966988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 868/1755] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:33.966988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:33.966988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:34.182389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:34.186083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:34.195521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca63c8-7962-42d1-91ec-11508bb56c0f service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:34.279293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 888/1756] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 879 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:34.447488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9bb4c-b28c-4146-b57e-b01b6b569592 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:34.459018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:34.462778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:34.463051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:34.463519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] PATH_INFO: `/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:34.465912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43041cf735a4ac98ce8ac651a4caf6e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:34.469555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:34.687722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43041cf735a4ac98ce8ac651a4caf6e', 'request_id': 'req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:34.718774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43041cf735a4ac98ce8ac651a4caf6e', 'request_id': 'req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:34.747071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43041cf735a4ac98ce8ac651a4caf6e', 'request_id': 'req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:34.791713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] The token cache is being invalidate because project f43041cf735a4ac98ce8ac651a4caf6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:34.792210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:34.798642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f90754b-d29d-4bd2-bebb-57fcaf6cf10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:34.813394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 869/1757] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:33 2022] DELETE /identity/v3/projects/f43041cf735a4ac98ce8ac651a4caf6e => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:34.819642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:34.820223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:34.820645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:34.884089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:34.886159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:35.244418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 889/1758] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:35.256275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:35.258489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:35.259256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:35.333278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:35.336722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:35.391897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:35.403849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 870/1759] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:35.424321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feaad6-221b-4b8a-8d6e-7809349d4db2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:35.468403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:35.472569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:35.476389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:35.477188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:35.482474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:35.487167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:35.495779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:35.565344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 890/1760] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:35.567348 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:35.618621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732034b9-19e0-4512-a351-b6ec3539318b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:35.635640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:35.637965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:35.638510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:35.638998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:35.641187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:35.660291 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:35.775210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 871/1761] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:35 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:39:35.779099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:35.814845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee35f-5be8-4c67-a0d7-0e0b08f35e77 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:35.894481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:36.135355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:36.136600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:36.137108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:36.137615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:36.139066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:36.151504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:36.227306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:36.246909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 891/1762] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:35 2022] POST /identity/v3/users => generated 291 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:39:36.281001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8c3b-ef57-44e6-95fa-c6ae61ea7afd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:36.313486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:36.314723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:36.315675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:36.316493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:36.318069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:36.326642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:36.389356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 872/1763] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:36.391492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:36.437322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb332a04-9795-48a1-8d8b-a216d66d738e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:36.475642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:36.490380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:36.491328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:36.492084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:36.493875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:36.508272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3200f9-4829-42ee-bfe5-0052652ab212 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:36.560764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 892/1764] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:36.566664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:36.603450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d2ada-cca8-4777-9a17-cd35e7911171 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:36.623067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:36.624324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:36.624904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:36.625512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] PATH_INFO: `/v3/projects/329fb99bed4947bab101b2739e3e351b/users/1f9164328bea4bdc812c7bbde72a88fd/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:36.780478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329fb99bed4947bab101b2739e3e351b, user_id=1f9164328bea4bdc812c7bbde72a88fd, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:36.814403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:36.927339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 873/1765] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:36 2022] PUT /identity/v3/projects/329fb99bed4947bab101b2739e3e351b/users/1f9164328bea4bdc812c7bbde72a88fd/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:36.939988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:36.940787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:36.941275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:37.046971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] MFA Rules not processed for user `1f9164328bea4bdc812c7bbde72a88fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:37.049159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:37.361279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 893/1766] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:37.370758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:37.699702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:37.758351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffecc-4857-423c-b3ff-8dd21ba78885 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:37.784830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:37.785575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:37.785865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:37.792484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:37.793627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:37.795641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:37.865097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:37.900486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 874/1767] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:37 2022] POST /identity/v3/projects => generated 379 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:39:37.928570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a377d-0d70-45ff-87d2-dcb32e6b7eda None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:38.159754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:38.160575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.161157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.161825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:38.163198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:38.170502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:38.243721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:38.245341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 894/1768] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:37 2022] POST /identity/v3/users => generated 296 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:38.265199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb54f2-53a9-466d-8a2d-8899a842a50a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:38.279840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:38.290645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.290645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.290645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:38.292303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:38.299331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:38.334974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 875/1769] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:38.342940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:38.406491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77281-19b0-49a5-9a2d-decc979394d7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:38.444268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:38.446160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.452183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.452955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] PATH_INFO: `/v3/projects/dc523ce8b2914b6da566f70cc9803e7a/users/454da26ea58d42b697728af8f646acc8/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:38.549053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:38.572380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc523ce8b2914b6da566f70cc9803e7a, user_id=454da26ea58d42b697728af8f646acc8, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:38.578038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:38.654997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 895/1770] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:38 2022] PUT /identity/v3/projects/dc523ce8b2914b6da566f70cc9803e7a/users/454da26ea58d42b697728af8f646acc8/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:38.656652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:38.691642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6bfbc-efcd-4f03-9c60-51efc457a566 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:38.804471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:38.805474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.805999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.806943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:38.808220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:38.810133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bc63c-83f5-46ee-a32f-1670e17111a8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:38.830326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:38.841876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 876/1771] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:38.845464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8f48-641d-48b9-afd5-4ce759844441 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:38.853080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:38.854310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.854922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.855599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] PATH_INFO: `/v3/projects/dc523ce8b2914b6da566f70cc9803e7a/users/454da26ea58d42b697728af8f646acc8/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:38.859446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc523ce8b2914b6da566f70cc9803e7a, user_id=454da26ea58d42b697728af8f646acc8, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:38.863763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b4271-28a8-4fbc-9868-18b0c9bd1adb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:38.895374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed211-3551-4274-bd5d-b8ac80284f9f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:38.896459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 896/1772] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:38 2022] PUT /identity/v3/projects/dc523ce8b2914b6da566f70cc9803e7a/users/454da26ea58d42b697728af8f646acc8/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:38.902055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed211-3551-4274-bd5d-b8ac80284f9f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:38.902545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed211-3551-4274-bd5d-b8ac80284f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.025851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85fed211-3551-4274-bd5d-b8ac80284f9f None None] MFA Rules not processed for user `454da26ea58d42b697728af8f646acc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:39.028003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85fed211-3551-4274-bd5d-b8ac80284f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.042605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddfe2-8fdf-4443-97c9-57c462eb480a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.043507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddfe2-8fdf-4443-97c9-57c462eb480a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.045836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddfe2-8fdf-4443-97c9-57c462eb480a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.050585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 897/1773] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:39:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:39.079207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.081557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.083659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.195916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:39.197371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.356380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 877/1774] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:39.357917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.358182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.358518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.443400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:39.446166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.551909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 898/1775] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:39.554219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:39.557370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.635169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:39.675367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 878/1776] 10.222.0.95 () {58 vars in 1015 bytes} [Mon Jul 18 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:39.676424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:39.695813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.735254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f79f4-672c-4372-8f32-5b0100e98c47 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:39.743371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:39.746891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.747472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.747916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.762994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:39.765882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:39.780055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 879/1777] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:39.781707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:39.788368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:39.807526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d12b0-e01d-4a5b-88a1-4883006ebf32 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:39.822768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:39.823889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.824396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.825015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.833686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc26a5-5f7b-4384-a8dd-7798dddd8983 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:39.839357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:39.841194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:39.841654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:39.842077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:39.855011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:39.857295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:39.862824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:39.868484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:39.891008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 880/1778] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:39.909261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 899/1779] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:41.668416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:41.710197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d1fff-74ed-4200-8852-06f4afa91873 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:41.743913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454da26ea58d42b697728af8f646acc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc523ce8b2914b6da566f70cc9803e7a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:41.744991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:41.745564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:41.749273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:41.750341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:41.752614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:41.754565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:41.781765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 881/1780] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:39:41.791542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:41.831810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05162851-771a-4652-b647-04a64aa6e139 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:41.864604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454da26ea58d42b697728af8f646acc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc523ce8b2914b6da566f70cc9803e7a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:41.873691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:41.874561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:41.879923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:41.887335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:41.919007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:42.120757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:42.120757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:42.127451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 900/1781] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:41 2022] POST /identity/v3/projects => generated 354 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:42.127451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:42.444649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:42.461421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb60d2-febe-4a58-8fcf-5148fea0cd00 tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:42.477238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b3fc-67d1-4915-9547-a706a38f747d tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:42.478226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454da26ea58d42b697728af8f646acc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc523ce8b2914b6da566f70cc9803e7a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:42.479569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:42.480029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:42.480356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:42.481144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:42.481465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:42.483212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:42.523674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 901/1782] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:42.675688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:42.690349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:42.694254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:42.695613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] PATH_INFO: `/v3/users/4a36240c9a3d48c2b17ec0f275073f22` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:42.700620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a36240c9a3d48c2b17ec0f275073f22)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:42.704707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:42.748558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:42.804722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c1b6c-86cc-4fc8-a4f2-31bf1b4266ce tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:42.836443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454da26ea58d42b697728af8f646acc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc523ce8b2914b6da566f70cc9803e7a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:42.849809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:42.850662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:42.850990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] PATH_INFO: `/v3/projects/9fbd983e6ae34d25bf2f2689ea2551dd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:42.856313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a36240c9a3d48c2b17ec0f275073f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:42.859524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9fbd983e6ae34d25bf2f2689ea2551dd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:42.862046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:42.888055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a36240c9a3d48c2b17ec0f275073f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:42.912842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:42.943173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a36240c9a3d48c2b17ec0f275073f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:42.947512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a36240c9a3d48c2b17ec0f275073f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:42.950327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dee1d758-7ee6-4abb-ace3-bce93ba5f141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a36240c9a3d48c2b17ec0f275073f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:42.991906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 882/1783] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:42 2022] DELETE /identity/v3/users/4a36240c9a3d48c2b17ec0f275073f22 => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:43.011026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:43.011658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:43.012092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:43.101832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:43.107613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:43.120367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fbd983e6ae34d25bf2f2689ea2551dd', 'request_id': 'req-7231f200-358b-4651-9bee-c809cd899b9e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:43.154594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fbd983e6ae34d25bf2f2689ea2551dd', 'request_id': 'req-7231f200-358b-4651-9bee-c809cd899b9e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:43.186514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fbd983e6ae34d25bf2f2689ea2551dd', 'request_id': 'req-7231f200-358b-4651-9bee-c809cd899b9e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:43.274978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] The token cache is being invalidate because project 9fbd983e6ae34d25bf2f2689ea2551dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:43.277089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:43.278861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:43.305207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 902/1784] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:42 2022] DELETE /identity/v3/projects/9fbd983e6ae34d25bf2f2689ea2551dd => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:43.404399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 883/1785] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:43.508920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:43.510781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:43.892846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f200-358b-4651-9bee-c809cd899b9e tempest-VolumeQuotaClassesTest-222712194 tempest-VolumeQuotaClassesTest-222712194-project-admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:43.918058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:43.932909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:43.933203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:43.939042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:43.996788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:44.011920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:44.057315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 903/1786] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:44.381693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:44.383533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:44.431964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40984a-9cbe-4715-bf47-c59f51eb8ff1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:44.511590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:44.515876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:44.522991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:44.523610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] PATH_INFO: `/v3/users/7ba87da3287d41dc9dff40e02c851030` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:44.535685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba87da3287d41dc9dff40e02c851030)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:44.576947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:44.592987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:44.596018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:44.715789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788496e-b7e0-4ee7-974e-3ebb90a15c4a service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:44.756934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:44.759522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:44.760205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:44.760919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] PATH_INFO: `/v3/users/1f9164328bea4bdc812c7bbde72a88fd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:44.766925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9164328bea4bdc812c7bbde72a88fd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:44.780091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:44.987592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9164328bea4bdc812c7bbde72a88fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.008918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba87da3287d41dc9dff40e02c851030'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.016934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9164328bea4bdc812c7bbde72a88fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.065258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:45.068172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba87da3287d41dc9dff40e02c851030'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.076980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9164328bea4bdc812c7bbde72a88fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.100953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9164328bea4bdc812c7bbde72a88fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.102030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d91e093-9eb2-4c56-b4a2-aa6ac3162987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9164328bea4bdc812c7bbde72a88fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.124132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 904/1787] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:44 2022] DELETE /identity/v3/users/1f9164328bea4bdc812c7bbde72a88fd => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:45.131961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:45.132369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:45.133103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:45.175818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba87da3287d41dc9dff40e02c851030'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.178584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba87da3287d41dc9dff40e02c851030'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.183250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba87da3287d41dc9dff40e02c851030'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:45.202149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:45.207947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:45.299529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 884/1788] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:44 2022] DELETE /identity/v3/users/7ba87da3287d41dc9dff40e02c851030 => generated 0 bytes in 919 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:45.467815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 905/1789] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:45.659952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:45.662441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:45.928873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91726f5-58df-45cd-a2e0-399bf31b69ec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:45.937382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:45.938133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:45.939058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:45.939058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:45.954248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:46.266791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:46.273796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:46.375921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 885/1790] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:47.300168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:47.302487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.389582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28633284-d664-4d40-9b84-97b2ed492c07 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:47.432513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:47.444175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.447680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.448625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] PATH_INFO: `/v3/projects/68e857db505b4759ae2afc9661c2f2b0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.456693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e857db505b4759ae2afc9661c2f2b0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:47.467406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:47.640858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e857db505b4759ae2afc9661c2f2b0', 'request_id': 'req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:47.661912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e857db505b4759ae2afc9661c2f2b0', 'request_id': 'req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:47.722398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e857db505b4759ae2afc9661c2f2b0', 'request_id': 'req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:47.777122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] The token cache is being invalidate because project 68e857db505b4759ae2afc9661c2f2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:47.777550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:47.779370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:47.781918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 906/1791] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:47 2022] DELETE /identity/v3/projects/68e857db505b4759ae2afc9661c2f2b0 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:47.788086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:47.790490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.791709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:47.793586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.837720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43378-2a87-4c45-a2b5-64d67130a5e6 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:47.986488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1ac4-4788-47ff-ab84-d2ad61bb4dc2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:48.007552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:48.008963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:48.009527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:48.010024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] PATH_INFO: `/v3/users/1eb2f46ba1fb498885c671280815d875` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:48.011997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb2f46ba1fb498885c671280815d875)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:48.014712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:48.068561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:48.178195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:48.182614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb2f46ba1fb498885c671280815d875'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.190278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:48.192514 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:48.196491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] PATH_INFO: `/v3/projects/9722e5dc37ca423592891a601f0a0632` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:48.207064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9722e5dc37ca423592891a601f0a0632)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:48.211535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb2f46ba1fb498885c671280815d875'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.218960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:48.265830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb2f46ba1fb498885c671280815d875'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.296258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb2f46ba1fb498885c671280815d875'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.299084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-865a3023-1e8d-49f8-b2c0-3f2d71668122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb2f46ba1fb498885c671280815d875'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.335140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 907/1792] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:47 2022] DELETE /identity/v3/users/1eb2f46ba1fb498885c671280815d875 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:48.444815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9722e5dc37ca423592891a601f0a0632', 'request_id': 'req-58ad9dfd-984f-4e41-b4ce-334c6f3be254'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.475379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9722e5dc37ca423592891a601f0a0632', 'request_id': 'req-58ad9dfd-984f-4e41-b4ce-334c6f3be254'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.499638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9722e5dc37ca423592891a601f0a0632', 'request_id': 'req-58ad9dfd-984f-4e41-b4ce-334c6f3be254'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.544278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] The token cache is being invalidate because project 9722e5dc37ca423592891a601f0a0632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:48.544881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.546338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58ad9dfd-984f-4e41-b4ce-334c6f3be254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:48.548888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 886/1793] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:47 2022] DELETE /identity/v3/projects/9722e5dc37ca423592891a601f0a0632 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:48.554922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568e71-b316-4c70-88e4-1b2711a230d0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:48.556981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568e71-b316-4c70-88e4-1b2711a230d0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:48.557781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568e71-b316-4c70-88e4-1b2711a230d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:48.627777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d9568e71-b316-4c70-88e4-1b2711a230d0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:48.635430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9568e71-b316-4c70-88e4-1b2711a230d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:48.779592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc97e9-52ac-4bf8-a046-38007584b4bf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:48.780113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc97e9-52ac-4bf8-a046-38007584b4bf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:48.781896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc97e9-52ac-4bf8-a046-38007584b4bf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:48.783024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 887/1794] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:39:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:48.982933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:48.984056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:48.984056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.004196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 908/1795] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:49.006357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:49.006734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:49.007015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.023145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:49.024434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:49.071781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:49.073065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:49.074972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 888/1796] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:49.075785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:49.124398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f429b-d286-45a7-b417-0bc82a7602ba None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:49.154297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:49.155389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:49.155675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:49.155947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.159272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:49.159272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:49.188751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:49.219054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 889/1797] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:49.220294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:49.243368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:49.319588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d855c-0d23-4378-a71c-15832e9fb3d2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:49.340286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:49.341451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:49.341946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:49.342382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] PATH_INFO: `/v3/projects/329fb99bed4947bab101b2739e3e351b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.344238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329fb99bed4947bab101b2739e3e351b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:49.349249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:49.368372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:49.382465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 909/1798] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:39:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:49.392627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:49.460583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600138f6-4ff4-45d2-bde0-5527210a71ac None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:49.494829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329fb99bed4947bab101b2739e3e351b', 'request_id': 'req-4a588b7b-f478-4e98-a076-9aab42281b37'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:49.512843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329fb99bed4947bab101b2739e3e351b', 'request_id': 'req-4a588b7b-f478-4e98-a076-9aab42281b37'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:49.532375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:49.536267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329fb99bed4947bab101b2739e3e351b', 'request_id': 'req-4a588b7b-f478-4e98-a076-9aab42281b37'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:49.573241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] The token cache is being invalidate because project 329fb99bed4947bab101b2739e3e351b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:49.573539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:49.574360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:49.576385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 890/1799] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:49 2022] DELETE /identity/v3/projects/329fb99bed4947bab101b2739e3e351b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:49.583446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:49.587528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:49.687163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:49.688570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:49.690899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:49.691555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.693646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:49.698735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:49.786439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 910/1800] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:49 2022] POST /identity/v3/projects => generated 371 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:49.788676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:49.792839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:49.840611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a588b7b-f478-4e98-a076-9aab42281b37 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:49.968347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149e739-b77a-4976-99f3-95a28791c039 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:49.981443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:49.982240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:49.982565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:49.982923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:49.984355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:49.990652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:50.016899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:50.019307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.020036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.021239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] PATH_INFO: `/v3/users/454da26ea58d42b697728af8f646acc8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.024152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] RBAC: Authorizing `identity:delete_user(user_id=454da26ea58d42b697728af8f646acc8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:50.027764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:50.038364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 911/1801] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:49 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:50.041170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:50.046218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.226858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:50.235919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454da26ea58d42b697728af8f646acc8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:50.274880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454da26ea58d42b697728af8f646acc8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:50.337564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd21db-f59c-4d08-a285-20784a4eecb1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:50.354693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:50.355605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.355950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.356264 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.366629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.415286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454da26ea58d42b697728af8f646acc8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:50.416859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:50.418656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:50.432470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454da26ea58d42b697728af8f646acc8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:50.442286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-16ad2c87-0e6b-4ac4-aff2-76b628b5105f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454da26ea58d42b697728af8f646acc8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:50.468134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 912/1802] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:50.469229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:50.523738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d53d-bb20-4c01-a155-0938dfd246ca service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:50.545474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 891/1803] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:49 2022] DELETE /identity/v3/users/454da26ea58d42b697728af8f646acc8 => generated 0 bytes in 958 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:50.548406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.558288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.561907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.581013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 892/1804] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:39:50 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:50.585064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:50.592729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.643762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:50.827793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:50.829874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.832636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.832636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.838276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:50.841439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db124bb-b896-408e-b399-570c419ed09e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:50.892117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 913/1805] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:50.895853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.897583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.899475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.943736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:50.947490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.954078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f28ee-e24d-4dd2-98aa-d4fda120cb58 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:50.982594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:50.982594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.982594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.982594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] PATH_INFO: `/v3/projects/396b1d78b1e4487eb57c5238aa2c8565` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.989124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] RBAC: Authorizing `identity:delete_project(project_id=396b1d78b1e4487eb57c5238aa2c8565)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:50.999811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:51.221018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 914/1806] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:39:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:51.221925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:51.228756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.242829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396b1d78b1e4487eb57c5238aa2c8565', 'request_id': 'req-09b84912-9b67-48f8-965b-65cc0db0812a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.259338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396b1d78b1e4487eb57c5238aa2c8565', 'request_id': 'req-09b84912-9b67-48f8-965b-65cc0db0812a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.273061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bac48-c028-4688-832d-2dd0695d317a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:51.275789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396b1d78b1e4487eb57c5238aa2c8565', 'request_id': 'req-09b84912-9b67-48f8-965b-65cc0db0812a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.302752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] The token cache is being invalidate because project 396b1d78b1e4487eb57c5238aa2c8565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:51.303639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.305168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.310083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 893/1807] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:50 2022] DELETE /identity/v3/projects/396b1d78b1e4487eb57c5238aa2c8565 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:51.313007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:51.327542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.506835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:51.526560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:51.530437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.531821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.532669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.534712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:51.540814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab8e00-e401-4bdb-a3bc-1c75480d8253 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:51.544902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84912-9b67-48f8-965b-65cc0db0812a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:51.556158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:51.557237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.557559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.557765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] PATH_INFO: `/v3/users/3e94fdb746514b5db97343af949426e3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.561798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e94fdb746514b5db97343af949426e3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:51.566855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:51.574564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 915/1808] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:51.577624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.578611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.579451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.618018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:51.620060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.638761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 916/1809] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:51.640523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:51.657032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.705349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af92ef5-01d5-4df9-86e4-37ec309f3ec2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:51.728565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:51.730628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e94fdb746514b5db97343af949426e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.732563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.736094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.738646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] PATH_INFO: `/v3/projects/7e0e3f53fdd7410abfc570e699c35422/users/41ce7f39878142bdb8ebfce8bf427ad1/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.755945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e94fdb746514b5db97343af949426e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.786898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e94fdb746514b5db97343af949426e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.789329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e94fdb746514b5db97343af949426e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.790494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0e3f53fdd7410abfc570e699c35422, user_id=41ce7f39878142bdb8ebfce8bf427ad1, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:51.791335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57d682d1-a938-47c8-8e0a-f795edfd50a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e94fdb746514b5db97343af949426e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:51.796839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee4f76-3b94-4496-af00-b957514979f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:51.827927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 917/1810] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:51 2022] PUT /identity/v3/projects/7e0e3f53fdd7410abfc570e699c35422/users/41ce7f39878142bdb8ebfce8bf427ad1/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:51.829786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4be0e-e884-4c42-92a0-6c76210a24e3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.830269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4be0e-e884-4c42-92a0-6c76210a24e3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.830841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4be0e-e884-4c42-92a0-6c76210a24e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.861516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 894/1811] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:51 2022] DELETE /identity/v3/users/3e94fdb746514b5db97343af949426e3 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:51.864648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:51.865274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:51.866206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:51.892736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7dc4be0e-e884-4c42-92a0-6c76210a24e3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:51.897589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7dc4be0e-e884-4c42-92a0-6c76210a24e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.948299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] MFA Rules not processed for user `41ce7f39878142bdb8ebfce8bf427ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:51.950828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:51.960962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:52.126197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 918/1812] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:52.129174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.129476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.129758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.168634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 895/1813] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:52.170145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.174544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.191539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:52.197527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.236638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 919/1814] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:52.239731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.283051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2727d5a-5779-4dbc-a811-d5ba6847ab98 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.283275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:52.301947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.303714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.305449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.306231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.311356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:52.311916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.316673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6062e-abe6-4037-9b3b-24be4299151a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.322871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.330155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.331569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.332093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.332778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.335454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.354021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.358473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.371006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 920/1815] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:52.374381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.377163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.379024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 896/1816] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:52.379880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.382614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.410711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36a8a4-3009-4572-9f30-0eb5612bdeea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.426284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.428437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.432706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.433284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.438053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.447735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.531658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 921/1817] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:52.535128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.535226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c32432-4198-49c3-89b5-f1abe7ca5a77 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.664018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e157706-02fe-46fd-b446-ff9e56e7b1d8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.685958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.686675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.686968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.687431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.687548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.688400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.688851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.689286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.690247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.692430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.728666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.733289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.749615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 922/1818] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:52.751125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.756188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 897/1819] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:52.757887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:52.778239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40518-23b2-4880-b20b-d5158053a220 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.901197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0054d1b-8b1f-4e0f-9a96-823bc750ce2f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:52.913648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.914720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.915123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.915610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.916957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.919945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.930540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:52.941480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:52.942792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.943346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.943845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.945646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:52.951296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:52.993308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 923/1820] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:53.016722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.040295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 898/1821] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/users => generated 287 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:53.046777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.079835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ce70-6ebd-4f38-a5c9-58f0997ad642 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.090175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f2b6-8e25-4a28-9fe7-ad789a0d389c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.095570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.096276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.096574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.096954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.097610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.100043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.104127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.105371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.105951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.106528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.108345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.113646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.118816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 899/1822] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:53.119828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.133636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36fee-c89b-4404-acab-d6adc743e9e5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.143913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.144766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.145124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.146840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.147060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.149180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.166373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 900/1823] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:53.168635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 924/1824] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:53.170760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.171330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.184117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ad2f-2130-4ecb-a942-8e2e826b7986 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.190815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.192767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.193332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.193695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] PATH_INFO: `/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d/users/98656c90c1544cb2ac17d2d2492a70d5/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.214031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1465af7-7607-4739-a09d-cb2f3fd39b0e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.216472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:53.227756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9164b1999c1649cea2c4d9c1ada6fc0d, user_id=98656c90c1544cb2ac17d2d2492a70d5, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.228903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.231456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.231715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.231965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] PATH_INFO: `/v3/projects/64c9b7fcbe664d5281d923ba81c62132/users/00ef10cafc584efd98212faa8c4c1a9d/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.253223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.295694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 901/1825] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:53 2022] PUT /identity/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d/users/98656c90c1544cb2ac17d2d2492a70d5/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:53.296727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c9b7fcbe664d5281d923ba81c62132, user_id=00ef10cafc584efd98212faa8c4c1a9d, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.298217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cd04d-28b0-4bfb-a977-86d0e82b6363 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.299562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.342243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 925/1826] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:53 2022] PUT /identity/v3/projects/64c9b7fcbe664d5281d923ba81c62132/users/00ef10cafc584efd98212faa8c4c1a9d/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:53.369990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.370285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.370593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.408249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] MFA Rules not processed for user `00ef10cafc584efd98212faa8c4c1a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:53.409839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:53.500906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 926/1827] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:53.502002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.536517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670a8ef-2d99-4aea-babf-e9a0d8f902c9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.590695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af35d8e-5309-47b2-adb7-179c60a4abb3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.600135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.601033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.601453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.602025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.603021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.608092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.652239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 927/1828] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:53 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:53.653640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.692693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:53.737842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.739298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.739963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.740675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.742677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.746978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.758244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bfa49-242d-453e-a244-7be06edc4718 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.764221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.765674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.765674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.765958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.766726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.768889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.776731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 902/1829] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:53.778702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.799006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7b902-4355-434e-934c-b3dd591cf454 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.800216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 928/1830] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:53 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:53.801148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.811390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.813179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.813738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.814199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] PATH_INFO: `/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d/users/98656c90c1544cb2ac17d2d2492a70d5/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.815063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84626bc-c984-4cd7-975e-2670e0958c42 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.819521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9164b1999c1649cea2c4d9c1ada6fc0d, user_id=98656c90c1544cb2ac17d2d2492a70d5, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.821810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.822633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.822991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.823473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.824356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.825578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9489b-7fc0-4ac5-b8f8-658b8663e221 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.826672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.846697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 929/1831] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:53.853037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 903/1832] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:53 2022] PUT /identity/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d/users/98656c90c1544cb2ac17d2d2492a70d5/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:53.854728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.883292 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6a153-55c1-48e4-b6b7-3d7288e23712 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.886736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.889435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.890179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.950781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] MFA Rules not processed for user `98656c90c1544cb2ac17d2d2492a70d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:53.952945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.035533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.037354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.037891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.038451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] PATH_INFO: `/v3/projects/661d13b1f59049dabe41d2cde3e31490/users/fb5d436fd5004c478015c2fb9bb75504/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.068824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=661d13b1f59049dabe41d2cde3e31490, user_id=fb5d436fd5004c478015c2fb9bb75504, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:54.072170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:54.119682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 930/1833] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:53 2022] PUT /identity/v3/projects/661d13b1f59049dabe41d2cde3e31490/users/fb5d436fd5004c478015c2fb9bb75504/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:54.122432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:54.173625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 904/1834] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:54.194496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:54.196539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.208247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:54.278255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8aec3-caa4-4c48-a4e8-242d5edaa9ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:54.284560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.285407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.285714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.285995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.286954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:54.289845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:54.309929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 931/1835] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:39:54.311243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:54.325673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96cca0-a487-45ee-9afc-c064a284ff6d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:54.332430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76326fa9-fa63-408d-9dcc-d84d4b82c7a8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:54.335276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.336574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.336862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.337139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] PATH_INFO: `/v3/projects/661d13b1f59049dabe41d2cde3e31490/users/fb5d436fd5004c478015c2fb9bb75504/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.340403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661d13b1f59049dabe41d2cde3e31490, user_id=fb5d436fd5004c478015c2fb9bb75504, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:54.349724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a719e-996c-4ab0-b34a-2ca8aa61f3d8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:54.364224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 932/1836] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:54 2022] PUT /identity/v3/projects/661d13b1f59049dabe41d2cde3e31490/users/fb5d436fd5004c478015c2fb9bb75504/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:54.375421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.375734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.375971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.407428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:54.414231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] MFA Rules not processed for user `fb5d436fd5004c478015c2fb9bb75504`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:54.417915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.433561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.435074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.435662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.436438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.565429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 933/1837] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:54.565429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:54.567124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.626526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:54.629856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:54.684201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b595d5-81bc-40e7-b7e5-c4382628062a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:54.690721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.691622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.692075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.692368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] PATH_INFO: `/v3/projects/dc523ce8b2914b6da566f70cc9803e7a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.695150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc523ce8b2914b6da566f70cc9803e7a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:54.696543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 905/1838] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:54.697364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:54.701223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:54.710312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.747975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9503957-8a0f-4466-a3f1-959401530b0b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:54.774736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc523ce8b2914b6da566f70cc9803e7a', 'request_id': 'req-67127d21-ef40-49ce-b622-b2af94c023c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:54.782603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc523ce8b2914b6da566f70cc9803e7a', 'request_id': 'req-67127d21-ef40-49ce-b622-b2af94c023c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:54.793081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc523ce8b2914b6da566f70cc9803e7a', 'request_id': 'req-67127d21-ef40-49ce-b622-b2af94c023c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:54.811933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] The token cache is being invalidate because project dc523ce8b2914b6da566f70cc9803e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:54.812465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:54.821210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67127d21-ef40-49ce-b622-b2af94c023c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:54.824662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 934/1839] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:54 2022] DELETE /identity/v3/projects/dc523ce8b2914b6da566f70cc9803e7a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:54.831781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fa5-c1b6-4102-864f-f099c4d9743b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.832239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fa5-c1b6-4102-864f-f099c4d9743b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.832531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3fa5-c1b6-4102-864f-f099c4d9743b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.883371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5e3e3fa5-c1b6-4102-864f-f099c4d9743b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:54.884703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5e3e3fa5-c1b6-4102-864f-f099c4d9743b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.908489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:54.910232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:54.911453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:54.912859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:54.917677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:54.940743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:54.997539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:55.004633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 935/1840] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:55.025779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.026304 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.026632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.066005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:55.071352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.076513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.080636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.097087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 936/1841] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:55.100220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.119233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 906/1842] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:55.132414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da0788-0be8-4144-800f-2bc6461fb22d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.146251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.147717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.148227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.149171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.150448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:39:55.151921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.155913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.194854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 937/1843] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:39:55.196728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.233550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336b893-caed-4ee4-bb6f-6d9572b5e9b8 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.249669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.250546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.251923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.252430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.253712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.259767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.366945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 907/1844] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:55 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:39:55.373321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.377802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.382156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.387298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab261923-0b2a-4f4c-bcd2-d679d2c68246 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.468849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.469752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.470071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.470356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.471952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.474867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.526254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 938/1845] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:55 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:55.535754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.580896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811ae05-e4a1-40f2-ae54-6fd90e92687a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.590291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525a99-f7ab-4bfb-9212-2ae95f704f7e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.591557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.592829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.593125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.593885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.615889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.633211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.634447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.634994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.635502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.649121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.655076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.697694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 939/1846] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:55.699254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.746796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a238fe-cc35-4fcb-b616-0d29923398c1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.776375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.779339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.779339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.779339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.780768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.789844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.829884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.831668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.845533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 940/1847] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:55.847138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.851852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.863922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 908/1848] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:55.867151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.906992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccda2-dafb-4e13-9e67-d4c473a467ca service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.916339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f0eb7-9339-4758-a881-8bfae8740458 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:55.934498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.935947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.936440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.939647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] PATH_INFO: `/v3/projects/4369e8672a11418ca97504bb62b40267/users/a3ce5acbabe4420fab395e4854cd5b81/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.950853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:55.952389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.952879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.953363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] PATH_INFO: `/v3/users/41ce7f39878142bdb8ebfce8bf427ad1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.960411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ce7f39878142bdb8ebfce8bf427ad1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.980189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4369e8672a11418ca97504bb62b40267, user_id=a3ce5acbabe4420fab395e4854cd5b81, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:55.983400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:55.985820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:56.013041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 909/1849] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:55 2022] PUT /identity/v3/projects/4369e8672a11418ca97504bb62b40267/users/a3ce5acbabe4420fab395e4854cd5b81/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:56.014344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:56.024617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.128131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:56.232957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ce7f39878142bdb8ebfce8bf427ad1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.248494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ce7f39878142bdb8ebfce8bf427ad1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.263795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:56.278876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64941f9-e663-4318-b474-01bbb5e03158 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:56.279774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ce7f39878142bdb8ebfce8bf427ad1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.294271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:56.295222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.296673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ce7f39878142bdb8ebfce8bf427ad1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.297035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.297371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] PATH_INFO: `/v3/projects/a03e8aab03ca435b9055494dd1bea015` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.298825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f2a3b05f-ba54-4783-9ba0-35e2db4268b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ce7f39878142bdb8ebfce8bf427ad1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.300746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03e8aab03ca435b9055494dd1bea015)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:56.310147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:56.319643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 941/1850] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:55 2022] DELETE /identity/v3/users/41ce7f39878142bdb8ebfce8bf427ad1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:56.320690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.321936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.328531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.413872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] MFA Rules not processed for user `a3ce5acbabe4420fab395e4854cd5b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:56.423992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.455216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03e8aab03ca435b9055494dd1bea015', 'request_id': 'req-d3e18972-86b1-4df2-a599-65ccd1185869'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.469946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03e8aab03ca435b9055494dd1bea015', 'request_id': 'req-d3e18972-86b1-4df2-a599-65ccd1185869'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.483814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03e8aab03ca435b9055494dd1bea015', 'request_id': 'req-d3e18972-86b1-4df2-a599-65ccd1185869'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.508139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] The token cache is being invalidate because project a03e8aab03ca435b9055494dd1bea015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:56.508645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.511003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d3e18972-86b1-4df2-a599-65ccd1185869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.519601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 910/1851] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:56 2022] DELETE /identity/v3/projects/a03e8aab03ca435b9055494dd1bea015 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:56.522933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.523555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.524057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.577130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:56.586863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.649337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 942/1852] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:56.650829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:56.654261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.800048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 911/1853] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:56.802036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:56.807874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.821512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6732b-1c94-4785-99ce-2f30c27ee3f1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:56.830396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:56.831481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.831813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.832112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] PATH_INFO: `/v3/users/6a57e0d0d2a64f07b2879f8f2b04867c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.834624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a57e0d0d2a64f07b2879f8f2b04867c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:56.837968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:56.865462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30cd66-2db7-4994-940c-b2b559101582 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:56.876846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:56.878002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.878365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.878684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.884987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:56.889496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:56.924687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a57e0d0d2a64f07b2879f8f2b04867c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.943642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a57e0d0d2a64f07b2879f8f2b04867c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:56.968535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 912/1854] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:39:56 2022] POST /identity/v3/projects => generated 393 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:56.973174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:56.976747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.986138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a57e0d0d2a64f07b2879f8f2b04867c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:57.002996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a57e0d0d2a64f07b2879f8f2b04867c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:57.004289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a57e0d0d2a64f07b2879f8f2b04867c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:57.008079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:57.025330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 943/1855] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:56 2022] DELETE /identity/v3/users/6a57e0d0d2a64f07b2879f8f2b04867c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:57.026696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:57.031888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:57.121611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:57.157592 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db03a78-c6a7-4653-8a5e-f7b829ae5159 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.284442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.285526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.285959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.286342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.290657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:57.290881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76844-46ef-427b-b43e-dcc60b63cebf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.301471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.303104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.303508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.303894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.306493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.312375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.325137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.329662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.368144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 913/1857] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:57.369792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 944/1857] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:39:57 2022] POST /identity/v3/users => generated 303 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:39:57.371294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:57.385390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f282-a5b3-412a-9d4e-c8fd9fc87416 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.396611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.397950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.398785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.399197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.400568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.404184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.428541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:57.442153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 914/1858] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:57.445674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1352f5-38fe-4940-8247-7caa4b6b36ae admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.453049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.454277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.454734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.455132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] PATH_INFO: `/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d/users/acfb024874104e42a91e1ad5709b44b3/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.476138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6dc26bf1204e0f81fda481dfdb045d, user_id=acfb024874104e42a91e1ad5709b44b3, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.479436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.511947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:57.513657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 945/1859] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:57 2022] PUT /identity/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d/users/acfb024874104e42a91e1ad5709b44b3/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:57.531154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842139-b40d-4ea1-a15e-b37c082aee3c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.712556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.713251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.713533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.713811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.714594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.716579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52644703-86c3-45db-aa5b-73eca80c709f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.734218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 915/1860] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:57.736273 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:57.754273 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2f63-0836-4fc8-8db2-c6c53f032526 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:57.764623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:57.766690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.767475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.768546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] PATH_INFO: `/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d/users/acfb024874104e42a91e1ad5709b44b3/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.774317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6dc26bf1204e0f81fda481dfdb045d, user_id=acfb024874104e42a91e1ad5709b44b3, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:57.780989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:57.817589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 946/1861] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:39:57 2022] PUT /identity/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d/users/acfb024874104e42a91e1ad5709b44b3/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:57.819708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:57.820120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:57.820514 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:57.865010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] MFA Rules not processed for user `acfb024874104e42a91e1ad5709b44b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:57.867831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.065868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:58.068621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.080312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 916/1862] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:58.252365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43806ed8-bc09-4899-9dd4-b4e2df863f77 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:58.257636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:58.258344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.258750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.259053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.274647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:58.283613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:58.284844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:58.286715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.320513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:58.342898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 947/1863] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:58.457824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a2b8a-180a-4405-94cc-4fc5a0e0f374 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:58.461271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:58.464154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.465679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:58.466672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.466960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.467284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] PATH_INFO: `/v3/projects/7e0e3f53fdd7410abfc570e699c35422` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.468718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0e3f53fdd7410abfc570e699c35422)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:58.471048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:58.510285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2060b5-612a-4b04-aa0b-86eca33546ee service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:58.538034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:58.541038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.541229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.541705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.558776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.565492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0e3f53fdd7410abfc570e699c35422', 'request_id': 'req-19415da2-d128-4301-ab66-2f0e692a28d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.586990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0e3f53fdd7410abfc570e699c35422', 'request_id': 'req-19415da2-d128-4301-ab66-2f0e692a28d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.607453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0e3f53fdd7410abfc570e699c35422', 'request_id': 'req-19415da2-d128-4301-ab66-2f0e692a28d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.649862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] The token cache is being invalidate because project 7e0e3f53fdd7410abfc570e699c35422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:58.650175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.651607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.658597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 917/1864] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:58 2022] DELETE /identity/v3/projects/7e0e3f53fdd7410abfc570e699c35422 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:39:58.659995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:58.663645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.729821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:58.763613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:58.766723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:58.790083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415da2-d128-4301-ab66-2f0e692a28d2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:58.790234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 948/1865] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:58.797605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:58.798109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.798487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.798748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] PATH_INFO: `/v3/users/98656c90c1544cb2ac17d2d2492a70d5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.800842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98656c90c1544cb2ac17d2d2492a70d5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:58.803706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:58.906174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:58.926888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98656c90c1544cb2ac17d2d2492a70d5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.947310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98656c90c1544cb2ac17d2d2492a70d5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.982138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98656c90c1544cb2ac17d2d2492a70d5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.984173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98656c90c1544cb2ac17d2d2492a70d5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:58.985480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a647954-5e64-4a4d-8509-056a04fe21dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98656c90c1544cb2ac17d2d2492a70d5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.010091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 918/1866] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:39:58 2022] DELETE /identity/v3/users/98656c90c1544cb2ac17d2d2492a70d5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:59.350668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:59.351272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b8e0-fc07-4d73-ab0d-1a7e520796b1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:59.351576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b8e0-fc07-4d73-ab0d-1a7e520796b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:59.351703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b8e0-fc07-4d73-ab0d-1a7e520796b1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:59.353371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 919/1867] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:39:59.354739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:59.367850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9295ed1-c8d2-4b9a-bb49-84540ed0adf1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:59.368194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9295ed1-c8d2-4b9a-bb49-84540ed0adf1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:59.368404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9295ed1-c8d2-4b9a-bb49-84540ed0adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:59.386194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5772b-a22e-4618-a785-27f6e9f21d25 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:59.405542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9295ed1-c8d2-4b9a-bb49-84540ed0adf1 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:59.406890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9295ed1-c8d2-4b9a-bb49-84540ed0adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:59.597281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:59.600494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:59.600808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:59.601097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] PATH_INFO: `/v3/projects/1731d3615dc54aa5a9513f8daf723d7c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:59.606809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1731d3615dc54aa5a9513f8daf723d7c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:59.616061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:59.682380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 920/1868] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:59.732748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1731d3615dc54aa5a9513f8daf723d7c', 'request_id': 'req-0812be7b-2e13-4312-ad42-c252a325d4f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.742315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1731d3615dc54aa5a9513f8daf723d7c', 'request_id': 'req-0812be7b-2e13-4312-ad42-c252a325d4f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.756629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1731d3615dc54aa5a9513f8daf723d7c', 'request_id': 'req-0812be7b-2e13-4312-ad42-c252a325d4f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.794144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] The token cache is being invalidate because project 1731d3615dc54aa5a9513f8daf723d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:39:59.794751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.796780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0812be7b-2e13-4312-ad42-c252a325d4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:39:59.801087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 949/1869] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:39:59 2022] DELETE /identity/v3/projects/1731d3615dc54aa5a9513f8daf723d7c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:39:59.808018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:59.808698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:59.809276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:59.857069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:59.861094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.095122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 921/1870] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:00.106643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.107404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.108832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.174133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.175908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.207969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:00.216597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.236757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 950/1871] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:00.243953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.265613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73236-4655-46fb-9ebd-a526cf5ddb57 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.275352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.276141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.276449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.276737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.277503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:00.277812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.279571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.303503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 951/1872] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:00.309640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.315501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:00.333148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145d84-b073-4c41-9f8b-aa2b621189f2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.343049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.344091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.344430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.344713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.345935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.351651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.355445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7f2e-6eaa-4972-8d23-40fbb90bbc34 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.370511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.371151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.373139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.373139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.376142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.395188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 952/1873] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:00.395188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.398253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.408167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.409803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.429624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 922/1874] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:00.431076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.452312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f52745-ef9e-4e2c-bdbf-b3a7dad82c5e service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.470374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:00.537136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a088ca8-963b-4dec-98ab-caa9620ca4c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.549902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.552655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.553374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.553894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] PATH_INFO: `/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.556677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9164b1999c1649cea2c4d9c1ada6fc0d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.560919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.568903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.569869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.570273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.570766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.571957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.574185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.613024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 923/1875] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:00.617531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.656850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c26fa4-fb0b-4105-a3d2-88021aa09205 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.676477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9164b1999c1649cea2c4d9c1ada6fc0d', 'request_id': 'req-96abca15-6008-42d5-a225-4e6c544d5c2f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:00.690617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9164b1999c1649cea2c4d9c1ada6fc0d', 'request_id': 'req-96abca15-6008-42d5-a225-4e6c544d5c2f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:00.706709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9164b1999c1649cea2c4d9c1ada6fc0d', 'request_id': 'req-96abca15-6008-42d5-a225-4e6c544d5c2f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:00.732702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] The token cache is being invalidate because project 9164b1999c1649cea2c4d9c1ada6fc0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:00.733000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:00.735450 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96abca15-6008-42d5-a225-4e6c544d5c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:00.740785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 953/1876] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:00 2022] DELETE /identity/v3/projects/9164b1999c1649cea2c4d9c1ada6fc0d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:00.751775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e62c-51d8-4895-952d-43c1ef8cef05 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.752635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e62c-51d8-4895-952d-43c1ef8cef05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.755013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e62c-51d8-4895-952d-43c1ef8cef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.795194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f333e62c-51d8-4895-952d-43c1ef8cef05 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:00.798531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f333e62c-51d8-4895-952d-43c1ef8cef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.806221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.807288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.807739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.808035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.808995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.811045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.819633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 954/1877] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:00.830176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.830627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.831218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 924/1878] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:00.831386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.835881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.838230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.854241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b347f84-0d9b-4c53-a0a4-aea95623816f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.860712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.861535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.861895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.862279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.862640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:00.863846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.864785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:00.865074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.884000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 955/1879] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:00.884260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 925/1880] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:00.885427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.885608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.900826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aa154-344a-4fc5-b219-0831a9e336fc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.904885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee9830-f87f-4a0c-87dc-5bc5a9c73d5e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.907845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.908889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.909267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.909672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] PATH_INFO: `/v3/projects/914e9efe1b574f749ec0caa4c27790fc/users/bdf51990b9ff4a6e8f7d3440d1baae66/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.913492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:00.914709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.915209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.915714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:00.917147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:00.917729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.922012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.929745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=914e9efe1b574f749ec0caa4c27790fc, user_id=bdf51990b9ff4a6e8f7d3440d1baae66, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:00.933081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14e1b0-0e93-4b17-a6eb-736dfd4afe15 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:00.950452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 926/1881] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:00.951757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:00.954059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 956/1882] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:00 2022] PUT /identity/v3/projects/914e9efe1b574f749ec0caa4c27790fc/users/bdf51990b9ff4a6e8f7d3440d1baae66/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:00.963433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707625-210b-40da-b304-a8a33020ee3d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:00.968362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:00.969068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:00.969699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.016063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] MFA Rules not processed for user `bdf51990b9ff4a6e8f7d3440d1baae66`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:01.018102 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:01.023066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.070584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.071396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.071849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.072301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.073507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.076057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.118692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 957/1883] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:01.122380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 927/1884] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:00 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:01.124009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.127278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.141503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492e1d4-3469-4a3d-a9b7-77bc496df00c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.175798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.238772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.239633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.240027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.240413 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.241278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.244150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.249752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099861-c5f9-4474-a5ad-a6028ff78049 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.258935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.259895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.260318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.260760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.261750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.265836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.303363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 958/1885] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:01 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:01.307439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.310531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 928/1886] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:01 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:01.314012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.339993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8db18b-b781-44f4-a65d-c8a1c0139eb8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.460265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115e5e1-46b2-432e-8ddd-4b0d5fc119b3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.466551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.467401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.467794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.468179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.469115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.472770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.523306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 959/1887] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:01 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:01.525213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.536277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.538950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.540019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.542202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.544177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.547946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.549404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88acd-df9a-406d-8c0c-41f1f2a5b19b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.562936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.564034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.564571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.565082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.566360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.570502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.571862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 929/1888] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:01.572939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.591990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7f207-d2a0-47e6-8f69-2d357ae10a61 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.601623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 960/1889] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:01.602690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.604876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.606104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.606696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.607123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.608742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.612528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.613390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160add1c-a2fe-4e4c-8988-c3f86e39ac54 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.624019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.625743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.626184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.626691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] PATH_INFO: `/v3/projects/915897f0b859428dba61ed8258ea8aa5/users/859bd33a9a6f4825b0d72544ff52930a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.640724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 930/1890] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:01.643753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.656370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=915897f0b859428dba61ed8258ea8aa5, user_id=859bd33a9a6f4825b0d72544ff52930a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.659791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.672720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202e8dc-0c34-43cc-92b8-b22c6bab7e58 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.683748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 961/1891] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:01 2022] PUT /identity/v3/projects/915897f0b859428dba61ed8258ea8aa5/users/859bd33a9a6f4825b0d72544ff52930a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:01.688738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.717336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.763487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.765204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.765649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.766203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] PATH_INFO: `/v3/projects/84beed38e1a444dcb7ee63de1bcb8c22/users/cdbf6ddc9fb045c3924ec73d31f9ccf2/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.798774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84beed38e1a444dcb7ee63de1bcb8c22, user_id=cdbf6ddc9fb045c3924ec73d31f9ccf2, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.804999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c56c7-128a-4c2a-aeff-66b29213ef34 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.835764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 931/1892] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:01 2022] PUT /identity/v3/projects/84beed38e1a444dcb7ee63de1bcb8c22/users/cdbf6ddc9fb045c3924ec73d31f9ccf2/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:01.836760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ac72-9da9-4b8a-bc25-6a72011913f7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.853867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.854173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.854522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.854902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.936892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] MFA Rules not processed for user `cdbf6ddc9fb045c3924ec73d31f9ccf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:01.939182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:02.014079 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.014813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.015090 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.015991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.016771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.018528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.056812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 962/1893] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:02.060664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 932/1894] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:02.062912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.065859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.084696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd99eb-6019-480d-a9a4-b9e002fccacf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.087662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610793f2-7a25-4839-bc27-d1326fba1a83 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.100379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.102699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.103286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.103841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] PATH_INFO: `/v3/projects/915897f0b859428dba61ed8258ea8aa5/users/859bd33a9a6f4825b0d72544ff52930a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.107436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.107897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.108183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.108458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.109219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=915897f0b859428dba61ed8258ea8aa5, user_id=859bd33a9a6f4825b0d72544ff52930a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.109401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.111423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.113306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d807d3-56c1-4505-8160-40c0f4ba1d30 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.127270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 963/1895] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:02 2022] PUT /identity/v3/projects/915897f0b859428dba61ed8258ea8aa5/users/859bd33a9a6f4825b0d72544ff52930a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:02.139469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.139800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.140029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.154788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 933/1896] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:02 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:02.156229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.172134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] MFA Rules not processed for user `859bd33a9a6f4825b0d72544ff52930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:02.173383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:02.245810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccfec1-8aea-4ab2-9366-33b96d9f3304 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.254055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.255281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.255778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.256287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.257647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.261601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.268747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 964/1897] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:02.281716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.283286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:02.295432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 934/1898] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:02 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:02.302503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.324756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904904be-6227-439f-88fe-306b354ba967 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.334018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.335261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.335772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.336270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.337908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.341465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.366043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 935/1899] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:02.373243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.376799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f4c4-7a7b-4cbe-887c-e1b2ec67f1ef None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.382719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.382985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.383142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.383423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.384396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:02.389032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086475-c184-4882-a22a-cc02d82d8822 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.397363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.398960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.399350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.400362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.400760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.401303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] PATH_INFO: `/v3/projects/513b77bce1fb4da0aa15d3f64402be5f/users/cbc231c2f1264e8ba8b75db2db1211e4/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.410625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 965/1900] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:02.430232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=513b77bce1fb4da0aa15d3f64402be5f, user_id=cbc231c2f1264e8ba8b75db2db1211e4, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.437255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.457947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 936/1901] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:02 2022] PUT /identity/v3/projects/513b77bce1fb4da0aa15d3f64402be5f/users/cbc231c2f1264e8ba8b75db2db1211e4/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:02.465080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.476908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eabfa1-e97c-47c1-a6bc-2fba22917fba service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.500671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:02.528357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.576808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.577954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.579418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.579914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.581245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.584517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.606074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 966/1902] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:02.611928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.632545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb378e65-e47b-411f-8b55-a3c169dee5fa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.639630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.640553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.640836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.641172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] PATH_INFO: `/v3/projects/513b77bce1fb4da0aa15d3f64402be5f/users/cbc231c2f1264e8ba8b75db2db1211e4/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.646499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] RBAC: Authorizing `identity:create_grant(project_id=513b77bce1fb4da0aa15d3f64402be5f, user_id=cbc231c2f1264e8ba8b75db2db1211e4, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.648842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dca1da-9b61-47cc-bcbe-905b522d595f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.659556 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd418ef-3127-4708-8964-7bf202b35d6c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.675517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 967/1903] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:02 2022] PUT /identity/v3/projects/513b77bce1fb4da0aa15d3f64402be5f/users/cbc231c2f1264e8ba8b75db2db1211e4/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:02.688396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.690522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.691195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.730483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] MFA Rules not processed for user `cbc231c2f1264e8ba8b75db2db1211e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:02.733107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:02.774977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.776259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.776582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.777448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:02.873462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 968/1904] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:02.884204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:02.885762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:02.893500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:02.895109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:02.900419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 937/1905] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:02.909126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb09c9-d5f8-4ca0-9350-9df7db8f12c2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:02.914314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:02.915327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:02.915812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:02.916268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:03.007556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:03.009647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2c40-316a-4824-bafd-61f89aa67bf7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:03.026494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 969/1906] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:04.492171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:04.494841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:04.640917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:04.690692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85291f-5b59-4ff8-9570-40b050fdb49a service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:04.712575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:04.718948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:04.718948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:04.719202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] PATH_INFO: `/v3/users/a3ce5acbabe4420fab395e4854cd5b81` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:04.736328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ce5acbabe4420fab395e4854cd5b81)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:04.741085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:04.859198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ce5acbabe4420fab395e4854cd5b81'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:04.876340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ce5acbabe4420fab395e4854cd5b81'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:04.915357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ce5acbabe4420fab395e4854cd5b81'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:04.916610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ce5acbabe4420fab395e4854cd5b81'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:04.919155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ce5acbabe4420fab395e4854cd5b81'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:04.969531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 938/1907] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:04 2022] DELETE /identity/v3/users/a3ce5acbabe4420fab395e4854cd5b81 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:04.971042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:04.971524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:04.971936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:04.996974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:04.998582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:05.035305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:05.113940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 970/1908] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:05.125478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:05.135525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:05.272427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab12d1-1ee2-4fe0-ba70-b95595aee594 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:05.286432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:05.287558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:05.289569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:05.290108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:05.316184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:05.321446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:05.353851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 939/1909] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:05.875522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:05.877029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:05.891786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038e3e3-fa10-44a7-9fb3-e2dc93884f08 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:05.899822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:05.901164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:05.901616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:05.902053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] PATH_INFO: `/v3/projects/4369e8672a11418ca97504bb62b40267` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:05.904175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4369e8672a11418ca97504bb62b40267)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:05.907420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:05.987035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4369e8672a11418ca97504bb62b40267', 'request_id': 'req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:05.998476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4369e8672a11418ca97504bb62b40267', 'request_id': 'req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.014377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4369e8672a11418ca97504bb62b40267', 'request_id': 'req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.034373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] The token cache is being invalidate because project 4369e8672a11418ca97504bb62b40267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:06.034868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.035818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.038439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 971/1910] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:05 2022] DELETE /identity/v3/projects/4369e8672a11418ca97504bb62b40267 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:06.042562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:06.053698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82e60-6f8f-4f7e-b12b-7e0ae4a3c9a2 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:06.136854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:06.137758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:06.138045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:06.138325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] PATH_INFO: `/v3/users/acfb024874104e42a91e1ad5709b44b3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:06.139864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfb024874104e42a91e1ad5709b44b3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:06.142324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:06.164333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:06.240312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfb024874104e42a91e1ad5709b44b3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.252963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfb024874104e42a91e1ad5709b44b3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.278021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfb024874104e42a91e1ad5709b44b3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.279357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfb024874104e42a91e1ad5709b44b3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.280351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51c7c817-b21a-42e9-af01-68f4d4a82c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfb024874104e42a91e1ad5709b44b3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:06.301522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 940/1911] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:06 2022] DELETE /identity/v3/users/acfb024874104e42a91e1ad5709b44b3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:07.024855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.026861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:07.045370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ee3ed-d9ec-4a01-a227-532aadfa5a2b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.153568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.154645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.154935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.155223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] PATH_INFO: `/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.156532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6dc26bf1204e0f81fda481dfdb045d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.158612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.215938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6dc26bf1204e0f81fda481dfdb045d', 'request_id': 'req-be722b27-67cf-40a8-a5db-d277fbaa6c0e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:07.218427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:07.226045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6dc26bf1204e0f81fda481dfdb045d', 'request_id': 'req-be722b27-67cf-40a8-a5db-d277fbaa6c0e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:07.244187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6dc26bf1204e0f81fda481dfdb045d', 'request_id': 'req-be722b27-67cf-40a8-a5db-d277fbaa6c0e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:07.277712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] The token cache is being invalidate because project fb6dc26bf1204e0f81fda481dfdb045d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:07.278014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:07.278766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be722b27-67cf-40a8-a5db-d277fbaa6c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:07.280699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 972/1912] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:07 2022] DELETE /identity/v3/projects/fb6dc26bf1204e0f81fda481dfdb045d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:07.289653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.291597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.292301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.325361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:07.326657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:07.429399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 941/1913] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:07.438513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.438810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.439097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.460078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:07.461917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:07.473349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 973/1914] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:07.480509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.498309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4ce6-ae57-4355-8986-a6ee35b2ada7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.506630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.507394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.507885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.508385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.509817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:07.510367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.513584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.537003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 942/1915] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:07.541822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.551924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8b63d-c60d-4d1d-bd3d-9dca884eaa1d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.557084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.557763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.558054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.558331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.559091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.561035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.591594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 974/1916] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:07.595948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.607795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e69b6-0a30-4d51-8696-bc30752bf0b0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.671783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.672459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.672762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.673008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.673717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.675708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.699724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 943/1917] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:07.704227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.715940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5fdc-4e04-4614-910c-fef73d922afa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.720926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.721614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.721907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.722186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.722981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.724692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.737733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 975/1918] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:07.742369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.753225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014768f-6d9d-49e7-bb42-d0d2e0d4a205 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.758612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.759300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.759584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.759864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.760608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.762255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f179937-0cde-4a9a-a36a-0cffd41e672c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.764956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:07.778593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 944/1919] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:07.782383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:07.793739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5377fd-80fb-4888-b8e5-6f9a907920c5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:07.798817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:07.799739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.800028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.800310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] PATH_INFO: `/v3/projects/e1ad8319feff487ea5b58e82d751c4bf/users/e837279476a442a2a67e207b336a9761/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.817220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ad8319feff487ea5b58e82d751c4bf, user_id=e837279476a442a2a67e207b336a9761, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:07.820535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:07.836733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 976/1920] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:07 2022] PUT /identity/v3/projects/e1ad8319feff487ea5b58e82d751c4bf/users/e837279476a442a2a67e207b336a9761/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:07.847699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:07.847985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:07.848332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:07.888199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] MFA Rules not processed for user `e837279476a442a2a67e207b336a9761`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:07.889482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:07.968931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 945/1921] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:07.973714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.049886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd18e5-5858-417e-a955-fa3be7b210c7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.059201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.060183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.060723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.061046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.062211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.064357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.097477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 977/1922] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:07 2022] POST /identity/v3/projects => generated 399 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:08.103493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.121965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ffdf5-e2f0-4f4d-8471-6787e9126949 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.237673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.238745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.239137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.240164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.242271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.245185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.294968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 946/1923] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:08 2022] POST /identity/v3/users => generated 306 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:08.295956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.306901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be0139-a4ef-4b35-a5b2-71ed5713d56b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.317064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.319239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.319899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.320537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.321656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.324809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.338334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 978/1924] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:08.342731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.357518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936f6a1-f8ec-416e-9988-fd965a2df80a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.362977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.364174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.365178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.365566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] PATH_INFO: `/v3/projects/24fa7e56db5e41c98eef308efad258b0/users/13d604323a234051836f7751af08b002/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.383713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fa7e56db5e41c98eef308efad258b0, user_id=13d604323a234051836f7751af08b002, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.386495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.406844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 947/1925] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:08 2022] PUT /identity/v3/projects/24fa7e56db5e41c98eef308efad258b0/users/13d604323a234051836f7751af08b002/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:08.413594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.424779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1752c30-efc9-42e3-b8fe-12097dde9134 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.478208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:08.492501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.493537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.493839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.494157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.494951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.496753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234ad3-88c5-45c9-aa11-0139774ad3be admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.509048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 979/1926] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:08.517183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.545370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c53e1-5405-415b-8586-ddf4acc14f71 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:08.564277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:08.565501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.565938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.566361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] PATH_INFO: `/v3/projects/24fa7e56db5e41c98eef308efad258b0/users/13d604323a234051836f7751af08b002/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.572630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fa7e56db5e41c98eef308efad258b0, user_id=13d604323a234051836f7751af08b002, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:08.581282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:08.622001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 948/1927] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:08 2022] PUT /identity/v3/projects/24fa7e56db5e41c98eef308efad258b0/users/13d604323a234051836f7751af08b002/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:08.624624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:08.625105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:08.625678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:08.695286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] MFA Rules not processed for user `13d604323a234051836f7751af08b002`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:08.697896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:08.864787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 980/1928] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:08.883517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:08.886401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:08.980281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:09.012058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e34ab-cf55-4cc9-82fb-d382e9c96b8d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:09.020397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:09.023158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:09.024715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:09.025657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:09.048959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:09.051975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:09.091715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 949/1929] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:09.233691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:09.253456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c127f0-7cd0-4694-a550-5fb4c6e45930 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:09.276902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:09.277780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:09.279313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:09.279691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:09.427460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:09.429217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:09.448812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 981/1930] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:09.530436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:09.534995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:09.554434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac860-80a5-4f1b-bdf3-2c9599273cf2 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:09.562093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:09.563364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:09.563956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:09.564548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:09.567906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:09.667361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:09.668817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375485d5-362b-4cd2-892b-45c9254c4533 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:09.679680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 950/1931] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:19.439378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=68 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:19.674557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=38 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:23.001693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:23.105512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c320583-ac51-4676-8868-deed65afb39c service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:23.114393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:23.115599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:23.116157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:23.116792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] PATH_INFO: `/v3/users/e837279476a442a2a67e207b336a9761` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:23.118688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e837279476a442a2a67e207b336a9761)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:23.121033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:23.186910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e837279476a442a2a67e207b336a9761'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:23.201171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e837279476a442a2a67e207b336a9761'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:23.218728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e837279476a442a2a67e207b336a9761'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:23.227067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e837279476a442a2a67e207b336a9761'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:23.227968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e837279476a442a2a67e207b336a9761'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:23.235340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 982/1932] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:23 2022] DELETE /identity/v3/users/e837279476a442a2a67e207b336a9761 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:23.240651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:23.241430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:23.241568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:23.264723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:23.267061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:23.364991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 951/1933] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:23.384451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:23.387293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:23.461319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826960ad-dcb2-49fd-9980-0aac930fd0cb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:23.465157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:23.465834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:23.466143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:23.466442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:23.477947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:23.479520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:23.492742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 983/1934] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:24.025947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:24.028098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:24.050944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb3635-ea7b-4ec2-bd4d-ae7c5a9098e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:24.061990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:24.063952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:24.066154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:24.068687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] PATH_INFO: `/v3/projects/e1ad8319feff487ea5b58e82d751c4bf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:24.073008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ad8319feff487ea5b58e82d751c4bf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:24.076573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:24.170667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ad8319feff487ea5b58e82d751c4bf', 'request_id': 'req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.201350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ad8319feff487ea5b58e82d751c4bf', 'request_id': 'req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.216571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ad8319feff487ea5b58e82d751c4bf', 'request_id': 'req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.243436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] The token cache is being invalidate because project e1ad8319feff487ea5b58e82d751c4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:24.244146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.244507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.253076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 952/1935] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:24 2022] DELETE /identity/v3/projects/e1ad8319feff487ea5b58e82d751c4bf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:24.253844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:24.276885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c442-ff66-4a57-aef2-d19f85d9840b service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:24.334667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:24.436173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:24.437636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:24.438124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:24.438630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] PATH_INFO: `/v3/users/13d604323a234051836f7751af08b002` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:24.440908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d604323a234051836f7751af08b002)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:24.445106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:24.541836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d604323a234051836f7751af08b002'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.552162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d604323a234051836f7751af08b002'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.604045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d604323a234051836f7751af08b002'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.634468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d604323a234051836f7751af08b002'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.634468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7e8d4bec-6e17-42e3-a557-c06525848cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d604323a234051836f7751af08b002'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:24.660011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 984/1936] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:24 2022] DELETE /identity/v3/users/13d604323a234051836f7751af08b002 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:24.976065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:24.984281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.016254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5ca2-0350-420c-bc1d-024dc00ebe62 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:25.161123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:25.163459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.164254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.164996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] PATH_INFO: `/v3/users/bdf51990b9ff4a6e8f7d3440d1baae66` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.168809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf51990b9ff4a6e8f7d3440d1baae66)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:25.174268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:25.242428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:25.272412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf51990b9ff4a6e8f7d3440d1baae66'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.280523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf51990b9ff4a6e8f7d3440d1baae66'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.296726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf51990b9ff4a6e8f7d3440d1baae66'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.297515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf51990b9ff4a6e8f7d3440d1baae66'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.298176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf51990b9ff4a6e8f7d3440d1baae66'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.331610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 953/1937] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:24 2022] DELETE /identity/v3/users/bdf51990b9ff4a6e8f7d3440d1baae66 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:25.336594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.337226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.337226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.347256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:25.355624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.369940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:25.371674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.451503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83736-e27a-4d0d-af07-107670d8697c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:25.457496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:25.459074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.459074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.459074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] PATH_INFO: `/v3/projects/24fa7e56db5e41c98eef308efad258b0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.460207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fa7e56db5e41c98eef308efad258b0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:25.462180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:25.505303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 985/1938] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:25.521421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:25.524331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.558463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fa7e56db5e41c98eef308efad258b0', 'request_id': 'req-72d631a3-ff1e-4760-ac5c-0203e6aa378c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.574350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fa7e56db5e41c98eef308efad258b0', 'request_id': 'req-72d631a3-ff1e-4760-ac5c-0203e6aa378c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.592223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fa7e56db5e41c98eef308efad258b0', 'request_id': 'req-72d631a3-ff1e-4760-ac5c-0203e6aa378c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.623890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] The token cache is being invalidate because project 24fa7e56db5e41c98eef308efad258b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:25.624706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.628780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72d631a3-ff1e-4760-ac5c-0203e6aa378c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:25.637545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 954/1939] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:25 2022] DELETE /identity/v3/projects/24fa7e56db5e41c98eef308efad258b0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:25.644040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7c18-28e9-4e77-9485-5a0949eaaf0b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.644650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7c18-28e9-4e77-9485-5a0949eaaf0b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.645160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7c18-28e9-4e77-9485-5a0949eaaf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.664895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:25.699948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2dcd7c18-28e9-4e77-9485-5a0949eaaf0b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:25.702329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2dcd7c18-28e9-4e77-9485-5a0949eaaf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.714332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d1a02-37f5-4c2e-9bf1-7d5a56ca635c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:25.770234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:25.825332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 955/1940] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:25.843615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.844029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.844719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.895931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:25.900757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:25.901165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.901968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.903204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.903770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.907678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:25.924108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 956/1941] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:25.928009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:25.929779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:25.930716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:25.943669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 986/1942] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:25.954438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8630dd-1a21-43ab-91ab-04b2303b4874 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:25.960617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:25.961623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:25.961934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:25.962179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:25.963792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:25.964162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:25.966353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:25.997802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 957/1943] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:26.002979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.020992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90512-5aae-4ce7-a85b-4c4f5ffaffdc service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.029540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.030720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.031363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.031926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.044393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.050600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.196648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 987/1944] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:26 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:26.198761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.232534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba93014-8271-41b0-8337-d09c82f0dd81 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.398327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.400532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.400979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.401263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.402304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.408467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.447189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 958/1945] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:26 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:26.452266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.470336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abdc9-4303-41b5-a83b-e16c25549f20 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.485833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.487657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.489285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.490158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.492041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.495404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.510575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 988/1946] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:26.516379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.529579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c030a0-ff75-4c6d-945a-b7554863cdb3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.535183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.536473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.537148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.537148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.538062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.539940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.554238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 959/1947] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:26.559904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.581122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3341-5e02-4c11-9282-ee59b64d0b9d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.586799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.592737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:26.597121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.600649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.601193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.601711 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] PATH_INFO: `/v3/projects/d51ed8525b214764af351a060fbdbfdb/users/ef7523f657c347b88b11cec931a349ae/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.626171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fd48-132d-467e-b23e-385806b11f22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.631350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51ed8525b214764af351a060fbdbfdb, user_id=ef7523f657c347b88b11cec931a349ae, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.636365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503f75-053b-4ceb-9a11-72688a3d9c45 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.636762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.638212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.638644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.639232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] PATH_INFO: `/v3/projects/914e9efe1b574f749ec0caa4c27790fc` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.641396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=914e9efe1b574f749ec0caa4c27790fc)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.645475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.664061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 989/1948] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:26 2022] PUT /identity/v3/projects/d51ed8525b214764af351a060fbdbfdb/users/ef7523f657c347b88b11cec931a349ae/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:26.678728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.679124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.679498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.712525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] MFA Rules not processed for user `ef7523f657c347b88b11cec931a349ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:26.713709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:26.721179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914e9efe1b574f749ec0caa4c27790fc', 'request_id': 'req-51aaee57-ed20-401e-8a91-ce29aa574d8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.731701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914e9efe1b574f749ec0caa4c27790fc', 'request_id': 'req-51aaee57-ed20-401e-8a91-ce29aa574d8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.741180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914e9efe1b574f749ec0caa4c27790fc', 'request_id': 'req-51aaee57-ed20-401e-8a91-ce29aa574d8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.758636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] The token cache is being invalidate because project 914e9efe1b574f749ec0caa4c27790fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:26.759166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.760491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.763781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 960/1949] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:26 2022] DELETE /identity/v3/projects/914e9efe1b574f749ec0caa4c27790fc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:26.767800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.772083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:26.803615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:26.804110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 990/1950] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:26.811543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.813034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:26.864599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:26.869820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaee57-ed20-401e-8a91-ce29aa574d8c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.876158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.877185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.877601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.877914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] PATH_INFO: `/v3/users/859bd33a9a6f4825b0d72544ff52930a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.879480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859bd33a9a6f4825b0d72544ff52930a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.881612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.916938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56fb2-ac79-4ac7-b6be-5e2f3fe6dfd9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:26.923421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:26.925278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:26.925278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:26.925278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:26.926099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:26.928362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:26.974493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859bd33a9a6f4825b0d72544ff52930a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:26.979714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 991/1951] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:26 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:26.991132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:26.996271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859bd33a9a6f4825b0d72544ff52930a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.029263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859bd33a9a6f4825b0d72544ff52930a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.031020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859bd33a9a6f4825b0d72544ff52930a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.032349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859bd33a9a6f4825b0d72544ff52930a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.067469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 961/1952] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:26 2022] DELETE /identity/v3/users/859bd33a9a6f4825b0d72544ff52930a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:27.133703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262821-5f37-4b13-850d-4b4b5cc63571 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.244157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.245210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.245632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.246142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.247317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.250703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.295361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 992/1953] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:26 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:27.297029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.298788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:27.319107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4fe20-2b3d-4741-87ad-b43805b76c34 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.351523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.353157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.353954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.354519 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.356412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.360027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.387471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 962/1954] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:27.392412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.409041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c7cc-be57-4d5d-bef5-a7e65835089c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.414909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.415702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.416058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.416407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.417248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.419049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e74b5e-1c43-4d63-b52d-f3ee6e373c7d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.433498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 993/1955] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:27.440809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.467109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ee9d2-07b2-474e-acae-eeb39e8d1d8f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.479574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.481506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.482190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.482912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] PATH_INFO: `/v3/projects/77b9b3e88e92497781e4f0737e328837/users/48720e2d11984403bbbbd1945a2134ff/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.518155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b9b3e88e92497781e4f0737e328837, user_id=48720e2d11984403bbbbd1945a2134ff, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.524503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.560475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 963/1956] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:27 2022] PUT /identity/v3/projects/77b9b3e88e92497781e4f0737e328837/users/48720e2d11984403bbbbd1945a2134ff/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:27.583165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.583824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.585311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.624798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] MFA Rules not processed for user `48720e2d11984403bbbbd1945a2134ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:27.626136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:27.678745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:27.738812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.741048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:27.743855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 994/1957] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:27.751120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.846037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6895a-2528-4181-9ab6-364cbe24a214 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.852342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.852907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.853186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.853458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] PATH_INFO: `/v3/projects/915897f0b859428dba61ed8258ea8aa5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.854754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=915897f0b859428dba61ed8258ea8aa5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.856726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.858666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:27.899214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e988fd4-969e-4f81-9184-e482890b5334 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:27.908213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:27.909542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.910104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.910434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:27.911323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:27.913452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:27.924325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915897f0b859428dba61ed8258ea8aa5', 'request_id': 'req-bd4661f8-fe30-4e27-bc54-b49040d41e4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.937932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915897f0b859428dba61ed8258ea8aa5', 'request_id': 'req-bd4661f8-fe30-4e27-bc54-b49040d41e4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.947814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915897f0b859428dba61ed8258ea8aa5', 'request_id': 'req-bd4661f8-fe30-4e27-bc54-b49040d41e4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.950254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 995/1958] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:27 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:27.954377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:27.966322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] The token cache is being invalidate because project 915897f0b859428dba61ed8258ea8aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:27.967952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.968737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bd4661f8-fe30-4e27-bc54-b49040d41e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:27.970825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 964/1959] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:27 2022] DELETE /identity/v3/projects/915897f0b859428dba61ed8258ea8aa5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:27.975830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250397e-88d2-4f1a-b115-6d140b9839ca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:27.976277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250397e-88d2-4f1a-b115-6d140b9839ca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:27.976817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250397e-88d2-4f1a-b115-6d140b9839ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.009325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0250397e-88d2-4f1a-b115-6d140b9839ca None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:28.012311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0250397e-88d2-4f1a-b115-6d140b9839ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:28.059501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714199-fd70-4a8a-b724-b867d695fc12 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.067439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.068322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.068707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.069085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.070803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.073527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.109485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 996/1960] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:27 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:28.114872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.117422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:28.126633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 965/1961] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:28.139357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.139726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.140095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.149921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb54c6a-e57a-4ded-93f5-bb7d6de8bf3b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.169868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.169868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.169868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.169868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.170359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:28.170472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.171990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:28.172515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.193501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 966/1962] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:28.195272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 997/1963] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:28.196409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.201948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.213560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef9006-19d0-463b-b91a-b977245bc7cd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.228897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.229967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.231200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.231410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698832b-ced4-401a-99b4-39c1601a9171 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.231781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.232298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:28.232684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.235945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.244027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.245306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.245914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.246564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] PATH_INFO: `/v3/projects/f4369ff23614493ba0137474483c03f1/users/9cfefc18040848ab8a6d6cf318bc665e/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.260496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 967/1964] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:28.268474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.284153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4369ff23614493ba0137474483c03f1, user_id=9cfefc18040848ab8a6d6cf318bc665e, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.291297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.292928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aff79b-8dcd-46e8-b18a-b7ae4c67cf08 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.306583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.308650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.309186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.311083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.313058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.320977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.325393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 998/1965] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:28 2022] PUT /identity/v3/projects/f4369ff23614493ba0137474483c03f1/users/9cfefc18040848ab8a6d6cf318bc665e/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:28.327201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.363866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:28.394751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 968/1966] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:28 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:28.399832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.472273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:28.494250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26abf3e-966a-451d-b668-8f2b234d8ea3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.510346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.511746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.512110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.512441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.514078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.516625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.524049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7513d2-51be-4e91-a48e-513176b75672 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.536414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.538066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.538878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.540000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.541637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.545458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.549448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 969/1967] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:28 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:28.550605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.563088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e56e-aa0b-4558-8f2b-464dd4af02da admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.570232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.571623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.571905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.572189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.573294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.574682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.575084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 999/1968] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:28.580309 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.590070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 970/1969] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:28.591709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.602044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142c5f-7616-473c-a1f3-5f5013435011 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.603942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3418-ffc2-4b91-84de-dfc9b7804d83 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.609669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.610146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.610658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.611038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.611416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.611604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.611952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.612387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] PATH_INFO: `/v3/projects/f4369ff23614493ba0137474483c03f1/users/9cfefc18040848ab8a6d6cf318bc665e/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.612610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.615234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.616321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4369ff23614493ba0137474483c03f1, user_id=9cfefc18040848ab8a6d6cf318bc665e, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.622093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54234f-330b-4a71-bf08-637083dde53c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.635853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 971/1970] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:28.637162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.646492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1000/1971] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:28 2022] PUT /identity/v3/projects/f4369ff23614493ba0137474483c03f1/users/9cfefc18040848ab8a6d6cf318bc665e/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:28.649599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8cf8-5d71-4d9b-a8c2-4e1d420d7e07 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:28.663494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.663494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.663925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.714116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] MFA Rules not processed for user `9cfefc18040848ab8a6d6cf318bc665e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:28.716461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:28.728037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:28.729125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.729469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.729852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] PATH_INFO: `/v3/projects/b194bc8bea7047afb7ccd58f0b04f364/users/5c99303682e044568e676ceb05be7cff/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.750459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b194bc8bea7047afb7ccd58f0b04f364, user_id=5c99303682e044568e676ceb05be7cff, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:28.753786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d286b-51e5-4f63-9ffa-ec0a5372a2d7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:28.782100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 972/1972] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:28 2022] PUT /identity/v3/projects/b194bc8bea7047afb7ccd58f0b04f364/users/5c99303682e044568e676ceb05be7cff/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:28.800684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:28.801895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:28.802287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:28.852556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1001/1973] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:28.875027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] MFA Rules not processed for user `5c99303682e044568e676ceb05be7cff`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:28.876962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:28.881331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:28.884142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:29.009053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 973/1974] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:29.021836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.031632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eaccc-3090-4412-996d-c0ea9ffccce5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.101674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:29.105685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:29.173751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.174730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.175239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.176400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.183469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b37c8-fffb-46a7-9692-a7b8a5260619 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.192800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.194707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.195199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.195634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.197375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.201189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.236328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 974/1975] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:29 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:29.241605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.360895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.364495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.388169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1002/1976] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:29.394472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fbdef-73f1-439d-b0eb-7efda9c2d794 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.403129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.403983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.404430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.404809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.406338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.409301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.452078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.460071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:29.462169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 975/1977] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:29 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:29.463995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.483461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb816c8-fc9a-42a6-8451-5a088e425000 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.492184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949c49-9754-4d78-be37-678dfc3fc9c7 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.494214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.496037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.496637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.497365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.499031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.502389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.534816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 976/1978] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:29.540418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.574259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba28db-9a1d-4380-b749-d9882dabf664 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.591216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.592682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.592962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.593291 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] PATH_INFO: `/v3/projects/cdf5b192cb114765afe72f9a67f1847d/users/c68b08ae10644bad8c1aa0f3610aff8a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.625112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf5b192cb114765afe72f9a67f1847d, user_id=c68b08ae10644bad8c1aa0f3610aff8a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.627104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.627841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.627992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.628123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.628363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.634533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:29.653057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 977/1979] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:29 2022] PUT /identity/v3/projects/cdf5b192cb114765afe72f9a67f1847d/users/c68b08ae10644bad8c1aa0f3610aff8a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:29.655043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.741697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.744171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.768440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1003/1980] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:29.803175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ec356-a586-49ab-bb30-2e54329ffd6e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.810187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.810919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.811202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.811479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.812356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.814953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552d155-b3be-4b07-936a-9873e8a8ab1c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.828883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 978/1981] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:29.834019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:29.868012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660f334-7349-4704-a728-521e8a38f8ec service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:29.883430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:29.884804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.885208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.886191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] PATH_INFO: `/v3/projects/cdf5b192cb114765afe72f9a67f1847d/users/c68b08ae10644bad8c1aa0f3610aff8a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.902446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf5b192cb114765afe72f9a67f1847d, user_id=c68b08ae10644bad8c1aa0f3610aff8a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:29.902446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:29.914438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1004/1982] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:29 2022] PUT /identity/v3/projects/cdf5b192cb114765afe72f9a67f1847d/users/c68b08ae10644bad8c1aa0f3610aff8a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:29.927300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:29.927825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:29.928341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:29.941039 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:29.979584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] MFA Rules not processed for user `c68b08ae10644bad8c1aa0f3610aff8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:29.980824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:30.180487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 979/1983] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:30.308739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:30.416458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:30.471609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96504e5-d82b-45f1-8c3b-39d6db0840e5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:30.477464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:30.478151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:30.478576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:30.478971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:30.562351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:30.563825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:30.575461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1005/1984] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:38.774328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:38.792793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:38.906696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3f8de-45ea-42af-8a0d-5211448eb050 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:38.916035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:38.917662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3005-e84b-497d-a956-9fc786820101 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:38.918105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:38.919041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:38.919675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] PATH_INFO: `/v3/users/5c99303682e044568e676ceb05be7cff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:38.922306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c99303682e044568e676ceb05be7cff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:38.926718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:38.928130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:38.928608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:38.929365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:38.930189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] PATH_INFO: `/v3/users/ef7523f657c347b88b11cec931a349ae` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:38.932892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7523f657c347b88b11cec931a349ae)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:38.937106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:39.026402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c99303682e044568e676ceb05be7cff'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.029828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7523f657c347b88b11cec931a349ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.038442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7523f657c347b88b11cec931a349ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.042173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c99303682e044568e676ceb05be7cff'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.055409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7523f657c347b88b11cec931a349ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.065593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7523f657c347b88b11cec931a349ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.067053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0882bd4c-dbc3-474f-bd1d-815fb50f5850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7523f657c347b88b11cec931a349ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.071667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c99303682e044568e676ceb05be7cff'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.072620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c99303682e044568e676ceb05be7cff'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.073552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7ed83b7-e8b8-444c-a1cc-ae7169728cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c99303682e044568e676ceb05be7cff'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.075302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1006/1985] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:38 2022] DELETE /identity/v3/users/ef7523f657c347b88b11cec931a349ae => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:39.080031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.080311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.080652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.087019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:39.098953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 980/1986] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:38 2022] DELETE /identity/v3/users/5c99303682e044568e676ceb05be7cff => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:39.104384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:39.105224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.105876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.106020 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.106692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.135686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:39.137495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.142883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:39.172819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1007/1987] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:39.185198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.186928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.214119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 981/1988] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:39.226171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.228564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.263142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1daa1c-3ea4-4ae8-9fc8-029cb48ecd87 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:39.269150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:39.270028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.270311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.270816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.290388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:39.293535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:39.305916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b38368-965a-4adc-8631-7656dfb60ad8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:39.309745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:39.310454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.310711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.310979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.314647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1008/1989] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:39.321255 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:39.322738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:39.334042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 982/1990] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:39.727998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.730085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.770152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd6a83-48c0-4f5f-bcd0-9da469372e80 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:39.777964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:39.779087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.779443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.779727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] PATH_INFO: `/v3/projects/d51ed8525b214764af351a060fbdbfdb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.782155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51ed8525b214764af351a060fbdbfdb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:39.786528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:39.800385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.802885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.818573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8335a-2387-4092-81e2-ed1acfd8e7ff service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:39.824028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:39.825203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:39.825366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:39.825655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] PATH_INFO: `/v3/projects/b194bc8bea7047afb7ccd58f0b04f364` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:39.827176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b194bc8bea7047afb7ccd58f0b04f364)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:39.829210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:39.859073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51ed8525b214764af351a060fbdbfdb', 'request_id': 'req-a538eed4-ef91-4746-ae1a-f8801244c4b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.866876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51ed8525b214764af351a060fbdbfdb', 'request_id': 'req-a538eed4-ef91-4746-ae1a-f8801244c4b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.877459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51ed8525b214764af351a060fbdbfdb', 'request_id': 'req-a538eed4-ef91-4746-ae1a-f8801244c4b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.897063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] The token cache is being invalidate because project d51ed8525b214764af351a060fbdbfdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:39.897589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.898615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.900658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1009/1991] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:39 2022] DELETE /identity/v3/projects/d51ed8525b214764af351a060fbdbfdb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:39.905041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.906781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.911058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b194bc8bea7047afb7ccd58f0b04f364', 'request_id': 'req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.919779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b194bc8bea7047afb7ccd58f0b04f364', 'request_id': 'req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.927242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b194bc8bea7047afb7ccd58f0b04f364', 'request_id': 'req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.941460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] The token cache is being invalidate because project b194bc8bea7047afb7ccd58f0b04f364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:39.941829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.942737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:39.944802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 983/1992] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:39 2022] DELETE /identity/v3/projects/b194bc8bea7047afb7ccd58f0b04f364 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:39.949104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:39.949590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:39.951162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:39.989573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538eed4-ef91-4746-ae1a-f8801244c4b5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:39.997761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:40.039619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13dfc76-8bd9-4f19-b1a9-8542ce298e9a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:40.047282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:40.048642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:40.049101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:40.049558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] PATH_INFO: `/v3/users/c68b08ae10644bad8c1aa0f3610aff8a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:40.051672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68b08ae10644bad8c1aa0f3610aff8a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:40.055363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:40.080185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:40.081585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:40.082721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:40.083183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] PATH_INFO: `/v3/users/48720e2d11984403bbbbd1945a2134ff` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:40.085008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48720e2d11984403bbbbd1945a2134ff)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:40.088754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:40.160685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68b08ae10644bad8c1aa0f3610aff8a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.176870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68b08ae10644bad8c1aa0f3610aff8a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.183739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48720e2d11984403bbbbd1945a2134ff'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.193392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48720e2d11984403bbbbd1945a2134ff'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.199249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68b08ae10644bad8c1aa0f3610aff8a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.200058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68b08ae10644bad8c1aa0f3610aff8a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.200720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68b08ae10644bad8c1aa0f3610aff8a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.222351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 984/1993] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:39 2022] DELETE /identity/v3/users/c68b08ae10644bad8c1aa0f3610aff8a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:40.230029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48720e2d11984403bbbbd1945a2134ff'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.245026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48720e2d11984403bbbbd1945a2134ff'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.246088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48720e2d11984403bbbbd1945a2134ff'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.254272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1010/1994] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:39 2022] DELETE /identity/v3/users/48720e2d11984403bbbbd1945a2134ff => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:40.595687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:40.597226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:40.712585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:40.715161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:40.724181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ae56a-b211-4c4f-9ca1-3125d313b6bd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:40.796456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:40.803497 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282dff9-75ca-439d-8864-8475b3412868 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:40.810381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:40.811586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:40.811959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:40.812516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] PATH_INFO: `/v3/projects/77b9b3e88e92497781e4f0737e328837` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:40.814877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b9b3e88e92497781e4f0737e328837)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:40.818084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:40.820406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:40.821371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:40.821756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:40.822081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] PATH_INFO: `/v3/projects/cdf5b192cb114765afe72f9a67f1847d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:40.823724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf5b192cb114765afe72f9a67f1847d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:40.825787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:40.828602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:40.885874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b9b3e88e92497781e4f0737e328837', 'request_id': 'req-1acadab4-552a-43f7-b954-7a4b26704795'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.893519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b9b3e88e92497781e4f0737e328837', 'request_id': 'req-1acadab4-552a-43f7-b954-7a4b26704795'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.901186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf5b192cb114765afe72f9a67f1847d', 'request_id': 'req-b469566c-99c4-4423-a20e-023d977e587a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.902832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b9b3e88e92497781e4f0737e328837', 'request_id': 'req-1acadab4-552a-43f7-b954-7a4b26704795'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.912807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf5b192cb114765afe72f9a67f1847d', 'request_id': 'req-b469566c-99c4-4423-a20e-023d977e587a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.917695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] The token cache is being invalidate because project 77b9b3e88e92497781e4f0737e328837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:40.917996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.918697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.920335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1011/1995] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:40 2022] DELETE /identity/v3/projects/77b9b3e88e92497781e4f0737e328837 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:40.920917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf5b192cb114765afe72f9a67f1847d', 'request_id': 'req-b469566c-99c4-4423-a20e-023d977e587a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.924655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:40.926777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:40.933537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] The token cache is being invalidate because project cdf5b192cb114765afe72f9a67f1847d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:40.933830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.934551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b469566c-99c4-4423-a20e-023d977e587a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:40.940025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 985/1996] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:40 2022] DELETE /identity/v3/projects/cdf5b192cb114765afe72f9a67f1847d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:40.941511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ea5ff-a984-4c8f-914c-1c2c33a6ea13 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:40.941801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ea5ff-a984-4c8f-914c-1c2c33a6ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:40.942081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ea5ff-a984-4c8f-914c-1c2c33a6ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:40.964463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1f9ea5ff-a984-4c8f-914c-1c2c33a6ea13 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:40.965980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f9ea5ff-a984-4c8f-914c-1c2c33a6ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.001088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acadab4-552a-43f7-b954-7a4b26704795 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.040960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 986/1997] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:41.050362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.050685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.051147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.071285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:41.072644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.075415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.076603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.076888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.077326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] PATH_INFO: `/v3/users/9cfefc18040848ab8a6d6cf318bc665e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.078814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cfefc18040848ab8a6d6cf318bc665e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.080794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.081430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 987/1998] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:41.086908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.107765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef994-2565-4ad6-aa08-c9e793244585 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.115734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.116799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.117268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.117749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.119145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:41.119797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.122964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.139471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:41.143075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 988/1999] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:40:41.150660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.154780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cfefc18040848ab8a6d6cf318bc665e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.164432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cfefc18040848ab8a6d6cf318bc665e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.164783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d16bd-d00a-4eed-87f2-1ee46589adb5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.170179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.171093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.171458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.171744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.172473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.174455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.180860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cfefc18040848ab8a6d6cf318bc665e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.191637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cfefc18040848ab8a6d6cf318bc665e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.192389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cfefc18040848ab8a6d6cf318bc665e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.200412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1012/2000] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:40 2022] DELETE /identity/v3/users/9cfefc18040848ab8a6d6cf318bc665e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:41.213229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 989/2001] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:41 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:41.217650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.219343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.307211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c016550-9386-4db3-abd2-35e2a75ec137 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.409111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.410311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.410851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.411357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.412739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.417760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.452734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1013/2002] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:41 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:41.457745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.477170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593e513-c452-46f7-99b6-ccb26bbcdf91 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.483189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.483884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.484209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.484468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.485950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.486864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.507436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 990/2003] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:41.509415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.522505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593b8ea-0b25-4793-8fb7-5c0412d53855 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.527833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.528423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.528705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.528992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.530481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.531480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549da08-b394-4495-9eb5-ab61f12cd53b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.546963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1014/2004] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:41.547885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.558791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f4c1-e193-40f5-9df8-46db2a346621 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.563642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.564568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.564844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.565142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] PATH_INFO: `/v3/projects/a907861f2aca4a4286e9a46cdae6c4d2/users/33e56d53506d49a68c864556486c94c4/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.581655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a907861f2aca4a4286e9a46cdae6c4d2, user_id=33e56d53506d49a68c864556486c94c4, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.584881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.609644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 991/2005] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:41 2022] PUT /identity/v3/projects/a907861f2aca4a4286e9a46cdae6c4d2/users/33e56d53506d49a68c864556486c94c4/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:41.615617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.615922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.616376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.640126 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:41.641925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.679784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.683080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.698743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:41.739772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:41.761316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1015/2006] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:41.775000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:41.777359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:41.825613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6636bf-17de-414b-84e1-dfe94f369fe2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.838779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.840066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.840562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.840858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] PATH_INFO: `/v3/projects/f4369ff23614493ba0137474483c03f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.842757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4369ff23614493ba0137474483c03f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.846250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.874080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6426430-1ba3-44dd-a969-23ba4b2db8ec None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:41.881809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:41.882978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.883753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.883991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:41.902124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:41.904054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3da509-5084-4fde-ab64-af1e917d801e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:41.917820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1016/2007] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:41.930361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4369ff23614493ba0137474483c03f1', 'request_id': 'req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.943744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4369ff23614493ba0137474483c03f1', 'request_id': 'req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.956703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4369ff23614493ba0137474483c03f1', 'request_id': 'req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.974333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] The token cache is being invalidate because project f4369ff23614493ba0137474483c03f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:41.974678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.975426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2ddc39f7-098c-4286-97a0-1ff4b5f6bdc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:41.982757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 992/2008] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:41 2022] DELETE /identity/v3/projects/f4369ff23614493ba0137474483c03f1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:41.984830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:41.985295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:41.985695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.023665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:42.026193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:42.155506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1017/2009] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:42.165682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.165960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.166246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.195939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:42.197356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:42.216335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 993/2010] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:42.217359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.231576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be6b11-9847-4374-b67e-abd3cb125fb9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.238866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.239848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.240157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.240446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.241201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:42.241509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.243250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.263110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1018/2011] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:40:42.269492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.286807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4089e8c-04c0-43e8-bcfd-f0670b62af5a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.294092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.294944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.295340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.295783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.296855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.299059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.386402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 994/2012] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:42 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:42.391661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.407406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d1b4-86f1-42fe-aa93-841681e4a8c4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.491287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.491974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.492281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.492570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.493409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.495313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.515142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:42.524384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1019/2013] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:42 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:42.527779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.539558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d6477-585a-4a3a-bc99-8d9568b565ac admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.544415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.545131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.545379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.545742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.546449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.548086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.566624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 995/2014] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:42.570075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.585473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea579-e9de-48e2-ab55-f8e8a762c9be admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.598461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.599434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.599729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.600072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.601029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.602742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f6a23-6a4a-485f-b6d5-e7339ee9e324 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.615910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1020/2015] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:40:42.623531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.637825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c062fac-26c5-4757-b3e5-5dfad720dac7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:42.643147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:42.644682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.645064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.645913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] PATH_INFO: `/v3/projects/81bbac01705345979247be7d4db666b6/users/242a4e59938a47fb9fa71b0e6d49c024/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.683150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bbac01705345979247be7d4db666b6, user_id=242a4e59938a47fb9fa71b0e6d49c024, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:42.686710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96860c0f-b5b0-45e6-a686-676cac3c455e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:42.713106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 996/2016] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:42 2022] PUT /identity/v3/projects/81bbac01705345979247be7d4db666b6/users/242a4e59938a47fb9fa71b0e6d49c024/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:42.732893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.733402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.733872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.797412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] MFA Rules not processed for user `242a4e59938a47fb9fa71b0e6d49c024`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:42.799653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:42.866980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:42.867952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:42.868380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:42.917371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] MFA Rules not processed for user `33e56d53506d49a68c864556486c94c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:42.918682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:42.927236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1021/2017] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:42.948459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:42.951505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:42.976722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:43.029474 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 997/2018] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:43.047054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:43.049866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:43.067312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cabb0-b04c-4835-bd1f-dd0c20cdd1a9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:43.071922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:43.072687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:43.073153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:43.073307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:43.087234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:43.088714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a8100-7481-4228-b064-642b84db19ca service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:43.101377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1022/2019] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:43.164391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca926-9efe-4e4c-a1ac-94366ef2f9fd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:43.168772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:43.169551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:43.169834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:43.171305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:43.180735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:43.182264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:43.193962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 998/2020] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:45.842030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f107f-05e8-4a3c-8715-4118e9d032c3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:45.843499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f107f-05e8-4a3c-8715-4118e9d032c3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:45.844081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f107f-05e8-4a3c-8715-4118e9d032c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:45.878741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d69f107f-05e8-4a3c-8715-4118e9d032c3 None None] MFA Rules not processed for user `33e56d53506d49a68c864556486c94c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:45.881382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d69f107f-05e8-4a3c-8715-4118e9d032c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:45.895930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1023/2021] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:46.807994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:46.810135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:46.910111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd8def-9503-4dfd-9bf7-d002b74e2bf4 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:46.918518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:46.919517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:46.919983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:46.920426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] PATH_INFO: `/v3/users/cdbf6ddc9fb045c3924ec73d31f9ccf2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:46.922662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbf6ddc9fb045c3924ec73d31f9ccf2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:46.927399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:47.044400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbf6ddc9fb045c3924ec73d31f9ccf2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.058279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbf6ddc9fb045c3924ec73d31f9ccf2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.083506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbf6ddc9fb045c3924ec73d31f9ccf2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.084506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbf6ddc9fb045c3924ec73d31f9ccf2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.085643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbf6ddc9fb045c3924ec73d31f9ccf2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.091539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:47.115342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 999/2022] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:46 2022] DELETE /identity/v3/users/cdbf6ddc9fb045c3924ec73d31f9ccf2 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:47.121775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:47.122612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:47.122612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:47.158751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:47.161067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:47.257135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1024/2023] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:47.275675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:47.277793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:47.385492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836a302-e12c-4b44-abf0-88fe623f3523 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:47.390677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:47.391491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:47.391874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:47.392245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:47.404277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:47.405751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:47.417396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1000/2024] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:47.764413 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:47.765917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:47.783691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d70d6-3c67-42f6-9185-22ce74bc405e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:47.789632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:47.790639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:47.790923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:47.791203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] PATH_INFO: `/v3/projects/84beed38e1a444dcb7ee63de1bcb8c22` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:47.792581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84beed38e1a444dcb7ee63de1bcb8c22)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:47.794676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:47.811047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:47.880238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84beed38e1a444dcb7ee63de1bcb8c22', 'request_id': 'req-f4a9ba71-2274-45b3-b755-ed114f84270f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.890802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84beed38e1a444dcb7ee63de1bcb8c22', 'request_id': 'req-f4a9ba71-2274-45b3-b755-ed114f84270f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.905166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84beed38e1a444dcb7ee63de1bcb8c22', 'request_id': 'req-f4a9ba71-2274-45b3-b755-ed114f84270f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.926827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] The token cache is being invalidate because project 84beed38e1a444dcb7ee63de1bcb8c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:47.927347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.928534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:47.931845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1025/2025] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:47 2022] DELETE /identity/v3/projects/84beed38e1a444dcb7ee63de1bcb8c22 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:47.936112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:47.947557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97102438-0fbb-4d09-b8e7-0dafce96abd9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:48.023600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:48.025143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:48.025612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:48.026083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] PATH_INFO: `/v3/users/cbc231c2f1264e8ba8b75db2db1211e4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:48.027614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc231c2f1264e8ba8b75db2db1211e4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:48.029756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:48.113050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc231c2f1264e8ba8b75db2db1211e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.122168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc231c2f1264e8ba8b75db2db1211e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.139536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc231c2f1264e8ba8b75db2db1211e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.140367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc231c2f1264e8ba8b75db2db1211e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.141254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1a82c31-7018-48d1-8304-7323e75a07aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc231c2f1264e8ba8b75db2db1211e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.157136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1001/2026] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:40:47 2022] DELETE /identity/v3/users/cbc231c2f1264e8ba8b75db2db1211e4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:48.517104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:48.520869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:48.544481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9ba71-2274-45b3-b755-ed114f84270f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:48.619140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:48.620983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:48.621472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:48.621943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] PATH_INFO: `/v3/projects/513b77bce1fb4da0aa15d3f64402be5f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:48.624157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=513b77bce1fb4da0aa15d3f64402be5f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:48.627533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:48.722640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513b77bce1fb4da0aa15d3f64402be5f', 'request_id': 'req-23e2aa83-bb2b-4255-a834-bf31461f30f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.731166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513b77bce1fb4da0aa15d3f64402be5f', 'request_id': 'req-23e2aa83-bb2b-4255-a834-bf31461f30f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.741168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513b77bce1fb4da0aa15d3f64402be5f', 'request_id': 'req-23e2aa83-bb2b-4255-a834-bf31461f30f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.761155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] The token cache is being invalidate because project 513b77bce1fb4da0aa15d3f64402be5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:40:48.761479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.762182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-23e2aa83-bb2b-4255-a834-bf31461f30f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:40:48.764531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1026/2027] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:40:48 2022] DELETE /identity/v3/projects/513b77bce1fb4da0aa15d3f64402be5f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:48.771297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:48.771806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:48.772339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:48.804705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:48.806010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:48.816625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:48.896657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1002/2028] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:48.906209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:48.906686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:48.907118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:48.934809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:48.936619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:48.948042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1027/2029] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:48.953219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:48.967235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6f59a-9a4c-4dc0-9d3a-cd06141889fc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:48.973740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:48.974601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:48.975062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:48.975460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:48.976359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:40:48.976852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:48.979213 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:48.994910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1003/2030] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:40:49.000776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.019934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6a0c-476c-48a9-8bd3-6badc90cbc0b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.045946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.049419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.049419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.049419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.049419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.050051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.111930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:49.127974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1028/2031] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:48 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:40:49.133516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.146516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6af3c-2bc8-4aff-8f52-05b389cf7980 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.211918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.212610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.212892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.213165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.213892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.215942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.239707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1004/2032] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:49 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:49.243958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.255365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a71bcb-6753-45b2-b926-7b622b5aaef6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.260259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.260937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.261216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.261493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.262233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.263905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.277017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1029/2033] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:49.282888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.294429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d3259-1d23-475d-b51f-197f321ea3fe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.299314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.300564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.300856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.301131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.301910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.303668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c51c2-3f42-4e09-ac09-0c3a176835fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.316481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1005/2034] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:49.323024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.334525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e1c92-70cd-4d61-af6f-1f99dfff78cf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.340158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.341166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.341570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.341923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] PATH_INFO: `/v3/projects/c3504db0f3b84d699d91a210c0762362/users/213af9e6523b43a9b4a6dc4fd5608a62/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.367133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3504db0f3b84d699d91a210c0762362, user_id=213af9e6523b43a9b4a6dc4fd5608a62, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.383200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.403045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1030/2035] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:49 2022] PUT /identity/v3/projects/c3504db0f3b84d699d91a210c0762362/users/213af9e6523b43a9b4a6dc4fd5608a62/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:40:49.408729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac388bb-ea2f-413f-8d39-89d29d199938 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.409060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac388bb-ea2f-413f-8d39-89d29d199938 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.409418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac388bb-ea2f-413f-8d39-89d29d199938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.429002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aac388bb-ea2f-413f-8d39-89d29d199938 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:49.430538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aac388bb-ea2f-413f-8d39-89d29d199938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:49.497149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1006/2036] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:40:49.508574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:49.511168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:49.587795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad9784-4dc7-4927-9dcf-fdc47c67ff9f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:49.592774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:49.593674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:49.594169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:49.594673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:49.607046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:49.608703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:49.620506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1031/2037] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:50.635199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:50.635780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:50.636083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:50.690955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] MFA Rules not processed for user `213af9e6523b43a9b4a6dc4fd5608a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:50.694745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:50.715188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:50.814168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1007/2038] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:50.820940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:50.833466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe76b0-3587-40d5-9246-db17a2e3cc85 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:50.846061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:50.846827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:50.847137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:50.847614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:50.848337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:50.850348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:50.891649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1032/2039] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:40:50 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:50.896848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:50.909315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ca544-c981-41d3-bd85-a578df17142e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:50.980294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:50.981403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:50.981403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:50.981719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:50.982430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:50.984444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:51.015702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1008/2040] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:40:50 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:40:51.016997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:51.035793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d58f1-2d71-4f00-8d4c-b7262eef9e07 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:51.045336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:51.046672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:51.046892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:51.047476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:51.049249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:51.055396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:51.084216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1033/2041] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:51.085642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:51.111426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36467c1-276c-472e-8b26-0e31c974c8d2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:51.120632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:51.124717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:51.124717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:51.124717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] PATH_INFO: `/v3/projects/67df224121d14a90b455960cfb919b72/users/23883be06ddd4ce7b99d2350666ace6c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:51.162451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] RBAC: Authorizing `identity:create_grant(project_id=67df224121d14a90b455960cfb919b72, user_id=23883be06ddd4ce7b99d2350666ace6c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:51.168949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:51.195944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1009/2042] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:51 2022] PUT /identity/v3/projects/67df224121d14a90b455960cfb919b72/users/23883be06ddd4ce7b99d2350666ace6c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:51.203017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:51.229324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0adeb4-f616-480e-b902-724377acb22f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:51.311543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:51.312352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:51.312789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:51.313167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:51.314031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:51.316495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8fe4d-7f6f-4f74-9883-fad9968740fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:51.327785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1034/2043] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:40:51.332772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:51.344019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4952174-89ac-4be7-ac6a-97a06c84acab admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:51.348919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:51.349841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:51.350126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:51.350398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] PATH_INFO: `/v3/projects/67df224121d14a90b455960cfb919b72/users/23883be06ddd4ce7b99d2350666ace6c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:51.352587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67df224121d14a90b455960cfb919b72, user_id=23883be06ddd4ce7b99d2350666ace6c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:51.355316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:51.368343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1010/2044] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:40:51 2022] PUT /identity/v3/projects/67df224121d14a90b455960cfb919b72/users/23883be06ddd4ce7b99d2350666ace6c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:40:52.278747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:52.279830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:52.280513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:52.340893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] MFA Rules not processed for user `23883be06ddd4ce7b99d2350666ace6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:52.343722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:52.387619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:52.482621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1035/2045] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:40:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:52.498563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:52.502059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:52.616693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af0ca-a014-47ab-84a4-e2736a8ef9a4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:52.621005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:52.621667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:52.622112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:52.622583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:52.647413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:52.717644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:52.719333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:52.730458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1011/2046] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:55.271530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:55.282955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ddea5-ebce-4c47-b8d2-13a5e02fb73a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:55.289121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:55.289859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:55.290105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:55.290359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:55.301927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:55.303451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece8044-9270-4f23-8d01-984a625e64bb service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:55.316111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1036/2047] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:02.725619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=19 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:05.310680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=27 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:13.247744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:13.249281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.339136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843895-a38d-45fa-a8d4-de50befefcbe service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:13.344601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:13.345658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.345949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.346229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] PATH_INFO: `/v3/users/00ef10cafc584efd98212faa8c4c1a9d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.347522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ef10cafc584efd98212faa8c4c1a9d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:13.349532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:13.430827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ef10cafc584efd98212faa8c4c1a9d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.439404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ef10cafc584efd98212faa8c4c1a9d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.453040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ef10cafc584efd98212faa8c4c1a9d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.453799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ef10cafc584efd98212faa8c4c1a9d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.454445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ef10cafc584efd98212faa8c4c1a9d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.471727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1012/2048] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:13 2022] DELETE /identity/v3/users/00ef10cafc584efd98212faa8c4c1a9d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:13.481030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.481576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.481576 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.513904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:13.515271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.612778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1037/2049] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:13.625552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:13.627987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.749154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989f301-7f1b-43e1-917e-6cdcb39def37 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:13.754866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:13.755770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.756205 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.756627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.775801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:13.777491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bb9b5-683d-40e9-9745-2d508019dfaf service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:13.792517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1013/2050] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:13.812847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:13.815242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.832928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da8848-7a32-4fc3-8c0a-3f46089fd786 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:13.838512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:13.839708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.840003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.840283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] PATH_INFO: `/v3/users/242a4e59938a47fb9fa71b0e6d49c024` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.841677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=242a4e59938a47fb9fa71b0e6d49c024)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:13.844767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:13.937178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242a4e59938a47fb9fa71b0e6d49c024'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.954159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242a4e59938a47fb9fa71b0e6d49c024'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.979432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242a4e59938a47fb9fa71b0e6d49c024'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.992595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242a4e59938a47fb9fa71b0e6d49c024'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:13.994118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242a4e59938a47fb9fa71b0e6d49c024'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.018238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1038/2051] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:13 2022] DELETE /identity/v3/users/242a4e59938a47fb9fa71b0e6d49c024 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:14.022678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:14.023240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:14.027562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:14.049380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:14.051378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:14.080962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:14.142689 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1014/2052] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:14.152597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:14.154050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:14.230357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8c7f-3943-48a7-a6b5-600e853adf9c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:14.234878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:14.235549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:14.235836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:14.236150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:14.251622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:14.254267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:14.269868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1039/2053] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:14.532632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:14.536973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:14.558322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f183300-f257-4548-ae74-70b7669b18ee None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:14.584543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:14.587714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:14.588521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:14.589614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] PATH_INFO: `/v3/projects/64c9b7fcbe664d5281d923ba81c62132` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:14.592975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c9b7fcbe664d5281d923ba81c62132)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:14.596991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:14.728880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c9b7fcbe664d5281d923ba81c62132', 'request_id': 'req-7c889cfe-2518-4d13-a5ec-e06732332758'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.748023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c9b7fcbe664d5281d923ba81c62132', 'request_id': 'req-7c889cfe-2518-4d13-a5ec-e06732332758'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.758843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c9b7fcbe664d5281d923ba81c62132', 'request_id': 'req-7c889cfe-2518-4d13-a5ec-e06732332758'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.780989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] The token cache is being invalidate because project 64c9b7fcbe664d5281d923ba81c62132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:14.781555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.786540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:14.792495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1015/2054] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:14 2022] DELETE /identity/v3/projects/64c9b7fcbe664d5281d923ba81c62132 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:14.794520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:14.810985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38831b-1de4-477d-9325-91534899d538 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:14.849140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:14.970370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:14.972621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:14.973191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:14.973466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] PATH_INFO: `/v3/users/fb5d436fd5004c478015c2fb9bb75504` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:14.975352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5d436fd5004c478015c2fb9bb75504)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:14.977131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:14.988287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:14.996230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:15.038982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c889cfe-2518-4d13-a5ec-e06732332758 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:15.052151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:15.057180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.059275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.059595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] PATH_INFO: `/v3/projects/81bbac01705345979247be7d4db666b6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.063177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bbac01705345979247be7d4db666b6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:15.065691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:15.133237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5d436fd5004c478015c2fb9bb75504'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.158773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5d436fd5004c478015c2fb9bb75504'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.208925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5d436fd5004c478015c2fb9bb75504'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.239207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5d436fd5004c478015c2fb9bb75504'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.240380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d95c8a-22b7-41b0-bd30-bc898b4f39cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5d436fd5004c478015c2fb9bb75504'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.261915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1040/2055] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:14 2022] DELETE /identity/v3/users/fb5d436fd5004c478015c2fb9bb75504 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:15.270244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bbac01705345979247be7d4db666b6', 'request_id': 'req-67f8098f-5b55-4e61-9400-e2e86872d185'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.286105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bbac01705345979247be7d4db666b6', 'request_id': 'req-67f8098f-5b55-4e61-9400-e2e86872d185'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.301163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bbac01705345979247be7d4db666b6', 'request_id': 'req-67f8098f-5b55-4e61-9400-e2e86872d185'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.321679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] The token cache is being invalidate because project 81bbac01705345979247be7d4db666b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:15.322591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.323647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67f8098f-5b55-4e61-9400-e2e86872d185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:15.332303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1016/2056] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:14 2022] DELETE /identity/v3/projects/81bbac01705345979247be7d4db666b6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:15.338006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.338525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.338907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.412607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:15.419774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:15.612774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1041/2057] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:15.623781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.626550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.627980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.653309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:15.654696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:15.673352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1017/2058] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:15.678770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:15.690447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff10f00-7b87-42bb-a199-bae49856d64f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:15.695796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:15.696838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.697317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.697803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.699098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:15.699605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:15.701815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:15.725971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1042/2059] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:15.730540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:15.742771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9462-f72f-4673-94f9-f0debccfe1f2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:15.750245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:15.750988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.751271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.751548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.752842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:15.755341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:15.791485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:15.807580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:15.809118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1018/2060] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:15 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:15.823785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eab2bb-4871-403f-a080-c6bff8b41419 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:15.850570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:15.899619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:15.901442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:15.930057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:15.930840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:15.931120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:15.931399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:15.934565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:15.934565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:15.979472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1043/2061] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:15 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:15.983626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:15.998327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451d54-0d89-4edd-983e-0fb5607fe549 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.006098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.007098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.007596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.008344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.009425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.014330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.038624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1044/2062] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:16.043736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.069140 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090df3a-c0da-4105-b772-58b4c15eabe1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.080962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.081667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.082201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.082523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.083482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.085142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.103423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1045/2063] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:16.104372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.112494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a03ca3-11cc-442c-a8a2-45c5248de067 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.118222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.119700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.120224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.120724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] PATH_INFO: `/v3/projects/661d13b1f59049dabe41d2cde3e31490` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.123506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=661d13b1f59049dabe41d2cde3e31490)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.124351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f814d-c6b4-4915-a60e-17739440d043 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.126290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.129699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.130652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.131165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.131529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] PATH_INFO: `/v3/projects/596431401ebd431492d52e760d961ede/users/0f069d8b472e4cb4807ada23855800c8/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.161747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=596431401ebd431492d52e760d961ede, user_id=0f069d8b472e4cb4807ada23855800c8, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.165849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1437ef-7442-4d41-925b-6518906a79f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.198689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1046/2064] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:16 2022] PUT /identity/v3/projects/596431401ebd431492d52e760d961ede/users/0f069d8b472e4cb4807ada23855800c8/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:16.200558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661d13b1f59049dabe41d2cde3e31490', 'request_id': 'req-5a645d48-e904-4638-ab62-af8ac198e59c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:16.210285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661d13b1f59049dabe41d2cde3e31490', 'request_id': 'req-5a645d48-e904-4638-ab62-af8ac198e59c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:16.220093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661d13b1f59049dabe41d2cde3e31490', 'request_id': 'req-5a645d48-e904-4638-ab62-af8ac198e59c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:16.235001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] The token cache is being invalidate because project 661d13b1f59049dabe41d2cde3e31490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:16.235398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:16.236025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5a645d48-e904-4638-ab62-af8ac198e59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:16.240600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1019/2065] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:15 2022] DELETE /identity/v3/projects/661d13b1f59049dabe41d2cde3e31490 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:16.247113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d0e1c-1cc0-4a5d-b092-27292c26c46c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.247113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d0e1c-1cc0-4a5d-b092-27292c26c46c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.247547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d0e1c-1cc0-4a5d-b092-27292c26c46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.260892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.262172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.262505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.286817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bf8d0e1c-1cc0-4a5d-b092-27292c26c46c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:16.288773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bf8d0e1c-1cc0-4a5d-b092-27292c26c46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:16.299959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] MFA Rules not processed for user `0f069d8b472e4cb4807ada23855800c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:16.301370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:16.393160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1020/2066] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:16.406192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.408138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:16.421538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1047/2067] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:16.431930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.432569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.432993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.473376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:16.495224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:16.496664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:16.513388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1048/2068] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:16.529434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.560951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f49c-55d8-4ea6-aed1-e22964488177 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.566788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.567486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.567829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.568119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.568892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:16.569205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.571455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.612685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1049/2069] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:16.614522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c3ca9-1e8e-40d7-8986-6cde857a2669 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.614716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.619165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.619835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.620126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.620407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.649148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.649148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.651849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f05-3fbb-43de-8559-cbc3af9ebb0a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.664800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.666518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.667153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.669857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.669857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.672357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.682719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1021/2070] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:16.731508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1050/2071] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:16.735682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:16.750746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee119-29a4-4455-b7d0-75de3f874983 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:16.935092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:16.935658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.935945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.936304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.937347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:16.941803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:16.989264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1022/2072] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/users => generated 284 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:16.990013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.004183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6f94-8dc8-4922-beac-b586fd87d751 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.010390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:17.011161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.011507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.011778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.012780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:17.014825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:17.039276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.057843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1051/2073] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:17.059740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a284-f990-494c-9aaf-7c11ff872e2b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.068484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:17.079727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.080056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.082093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.084881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:17.091800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692b46d-46d9-4541-9f4d-fcf281c1cbb5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:17.135328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1023/2074] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:17.141169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.182477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8469197-5cdc-42f0-866e-24250ac0f17b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.204002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:17.204870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.205238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.205998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] PATH_INFO: `/v3/projects/a8c9632f73b94362a561966e2691e7b2/users/b334f6d4b8cf442399062477d5d2d7cf/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.231385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:17.253175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c9632f73b94362a561966e2691e7b2, user_id=b334f6d4b8cf442399062477d5d2d7cf, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:17.256011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:17.287965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1052/2075] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:17 2022] PUT /identity/v3/projects/a8c9632f73b94362a561966e2691e7b2/users/b334f6d4b8cf442399062477d5d2d7cf/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:17.321293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.321293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.321615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.392973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] MFA Rules not processed for user `b334f6d4b8cf442399062477d5d2d7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:17.393721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:17.670447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1024/2076] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:17.671973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.817750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb16ba-a29d-49ec-a1d1-4e47e8c80745 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.823659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:17.824779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.824943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.825477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.826279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:17.828688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:17.868598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1053/2077] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:17 2022] POST /identity/v3/projects => generated 367 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:17.874186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.893151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c975fc5-3eee-4f9e-b7fc-fe752066cc34 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.908479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:18.040975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.041962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.042571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.042948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.043727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.045903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18433717-5839-4858-97f9-5a825b82b0d8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.078222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1025/2078] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:17 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:18.082777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.106452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.107984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.108369 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.110949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa369891-42a6-469b-b25c-fdff5354da76 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.113518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1026/2079] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:41:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:18.132448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.134445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.135881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.136266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.137202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.139255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe00a-b01d-44c7-ad70-bb08d55c01ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.144657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.150150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:18.158759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1054/2080] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:18.160460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.160979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.161483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.169935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41919754-8965-48fc-96ea-bc459a8dde75 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.176597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.177950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.178231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.178793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] PATH_INFO: `/v3/users/4a0fa5ed18f2499488e01dd95a65194d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.180367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0fa5ed18f2499488e01dd95a65194d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.183054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.190915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] MFA Rules not processed for user `52152797f2784be389bfae5dd38ef5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:18.192768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:18.259867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0fa5ed18f2499488e01dd95a65194d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:18.272615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0fa5ed18f2499488e01dd95a65194d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:18.315970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0fa5ed18f2499488e01dd95a65194d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:18.317058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0fa5ed18f2499488e01dd95a65194d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:18.318014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0fa5ed18f2499488e01dd95a65194d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:18.338374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1027/2081] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:18 2022] DELETE /identity/v3/users/4a0fa5ed18f2499488e01dd95a65194d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:18.339431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.343400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:18.364340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1055/2082] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:18.389662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.391092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:18.411275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9654f-4104-433d-91d6-904bc4779e4d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.541199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122dc09-5723-4297-886f-1612da05c413 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.557667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:18.557920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.559111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.559506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.559961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] PATH_INFO: `/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96/users/9978dfa266584eaf8dc3336399bd7c80/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.574847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.575873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.577114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.577909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.597661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b08a0d8798e4dfbbe6ad25802ce0d96, user_id=9978dfa266584eaf8dc3336399bd7c80, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.604859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.606656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.606977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.615504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:18.618952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1056/2083] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:18.638525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1028/2084] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:18 2022] PUT /identity/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96/users/9978dfa266584eaf8dc3336399bd7c80/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:18.644775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.667796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835a8c-4cfd-4281-b1d8-37374bb98c79 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.744457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.761144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:18.915799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41501b-f294-4900-91b1-543320521419 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.924416 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.925116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.925459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.925747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.926538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.928188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.937425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.938992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.939801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.940085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] PATH_INFO: `/v3/projects/49d4c19f14be46208297e4e3fb854bb0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:18.941486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d4c19f14be46208297e4e3fb854bb0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:18.944837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:18.957014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1057/2085] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:18.958401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:18.979770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f360b5-3f4c-4fa9-bbbb-3d84f9c173b1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:18.992541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:18.994876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:18.995166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:18.995418 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] PATH_INFO: `/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96/users/9978dfa266584eaf8dc3336399bd7c80/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.000995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b08a0d8798e4dfbbe6ad25802ce0d96, user_id=9978dfa266584eaf8dc3336399bd7c80, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:19.005190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc35c6-11c1-4d2f-977d-55a7168a192e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:19.030495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1058/2086] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:18 2022] PUT /identity/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96/users/9978dfa266584eaf8dc3336399bd7c80/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:19.030825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d4c19f14be46208297e4e3fb854bb0', 'request_id': 'req-c3e84c6f-0b3c-431c-adc9-a7552beff33d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.036773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.037384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.037898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.069494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d4c19f14be46208297e4e3fb854bb0', 'request_id': 'req-c3e84c6f-0b3c-431c-adc9-a7552beff33d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.087558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d4c19f14be46208297e4e3fb854bb0', 'request_id': 'req-c3e84c6f-0b3c-431c-adc9-a7552beff33d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.095709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] MFA Rules not processed for user `9978dfa266584eaf8dc3336399bd7c80`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:19.097859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.121496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] The token cache is being invalidate because project 49d4c19f14be46208297e4e3fb854bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:19.122332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.123810 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3e84c6f-0b3c-431c-adc9-a7552beff33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.127186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1029/2087] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:18 2022] DELETE /identity/v3/projects/49d4c19f14be46208297e4e3fb854bb0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:19.132493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.133618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.134002 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.163027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:19.165011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.240289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1059/2088] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:19.254070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.258334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.283809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1030/2089] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:19.295257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.297427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.324662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bc6bd-7289-4aec-a55b-aa5ec1ef9230 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:19.331669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:19.332928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.333319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.333705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] PATH_INFO: `/v3/users/213af9e6523b43a9b4a6dc4fd5608a62` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.335229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213af9e6523b43a9b4a6dc4fd5608a62)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:19.337600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:19.379953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:19.382282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbecace-5601-4ca6-96ba-4bcdcddec073 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:19.395774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:19.396487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.396809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.397118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.416964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:19.418935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e64f4-b71a-4049-8961-ddf1f67bcd4b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:19.427570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:19.436365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1060/2090] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:19.438812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.440441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.440441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.464504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:19.466222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.475306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213af9e6523b43a9b4a6dc4fd5608a62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.482643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1061/2091] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:19.491142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.513314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213af9e6523b43a9b4a6dc4fd5608a62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.521594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3307e-8c13-4777-9a7f-21164e53dd21 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:19.536171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:19.537338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.538094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.538997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.540625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:19.541347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:19.544670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:19.559643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213af9e6523b43a9b4a6dc4fd5608a62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.560491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213af9e6523b43a9b4a6dc4fd5608a62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.561708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213af9e6523b43a9b4a6dc4fd5608a62'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:19.570762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1062/2092] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:19.577952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.582825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:19.598957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1031/2093] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:19 2022] DELETE /identity/v3/users/213af9e6523b43a9b4a6dc4fd5608a62 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:19.625400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc747e50-f7dc-4061-832d-0db7daf6ef8f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:19.797313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:19.798242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:19.799321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:19.799734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:19.800489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:19.803283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:19.850332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1063/2094] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:19 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:19.856245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.873179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbd973-96b2-4aea-b307-3ca1cd4d2407 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:19.998810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:19.998810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.009420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.013227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.013524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.013806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.014564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.020394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.038659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ffd0-d305-4b13-8282-e819308ba645 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.063765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.065316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.065599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.065898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] PATH_INFO: `/v3/projects/c3504db0f3b84d699d91a210c0762362` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.072085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3504db0f3b84d699d91a210c0762362)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.072564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1032/2095] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:19 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:20.073604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.074198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.115005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e772e-54a6-485f-b666-4f07cc6a5ae7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.135114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.137209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.137655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.138331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.141847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.147865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.194780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1033/2096] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:20.197742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.223640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e9a4-45dc-4d3e-bbc6-b96ff5e809ec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.234598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.235544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.235716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.235878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.242707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.242707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.253803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1034/2097] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:20.258759 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.279220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a480327-67c6-4927-bd64-e2c217bfdceb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.290861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3504db0f3b84d699d91a210c0762362', 'request_id': 'req-ac37fe6f-7f31-4a3d-b1a7-d65259344059'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.299906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3504db0f3b84d699d91a210c0762362', 'request_id': 'req-ac37fe6f-7f31-4a3d-b1a7-d65259344059'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.313397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3504db0f3b84d699d91a210c0762362', 'request_id': 'req-ac37fe6f-7f31-4a3d-b1a7-d65259344059'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.329305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] The token cache is being invalidate because project c3504db0f3b84d699d91a210c0762362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:20.330265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.331279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.333151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1064/2098] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:19 2022] DELETE /identity/v3/projects/c3504db0f3b84d699d91a210c0762362 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:20.337721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.339441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.377829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.378806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.379092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.379428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] PATH_INFO: `/v3/projects/7291137bcc13437884d14bc24d1f67a4/users/2acccf73077e43ca84171315cc463acc/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.402667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:20.409213 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7291137bcc13437884d14bc24d1f67a4, user_id=2acccf73077e43ca84171315cc463acc, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.418250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b124-5946-4c9f-9f91-ac142e0127ca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.440248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1035/2099] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:20 2022] PUT /identity/v3/projects/7291137bcc13437884d14bc24d1f67a4/users/2acccf73077e43ca84171315cc463acc/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:20.450215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.450326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.450642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.452368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:20.481060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37fe6f-7f31-4a3d-b1a7-d65259344059 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.487335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] MFA Rules not processed for user `2acccf73077e43ca84171315cc463acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:20.488855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.584092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.585493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.585795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.586081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] PATH_INFO: `/v3/users/23883be06ddd4ce7b99d2350666ace6c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.588115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23883be06ddd4ce7b99d2350666ace6c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.590086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.598613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1036/2100] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:20.604409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.609962 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.632003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401bb0-fe1b-4947-bca3-cf30f1b1a1c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.637579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.638362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.638743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.639140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.647386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.657711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.695775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23883be06ddd4ce7b99d2350666ace6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.703230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1037/2101] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:20 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:20.707367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.721850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23883be06ddd4ce7b99d2350666ace6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.742728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23883be06ddd4ce7b99d2350666ace6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.755969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23883be06ddd4ce7b99d2350666ace6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.757026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23883be06ddd4ce7b99d2350666ace6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:20.769901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1065/2102] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:20 2022] DELETE /identity/v3/users/23883be06ddd4ce7b99d2350666ace6c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:20.862634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9c59a-323b-4912-9fb0-f1c183931d22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:20.972237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:20.972237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.972237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.972237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.973167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:20.973167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:20.980796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:21.008192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1038/2103] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:20 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:21.009350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.011481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.030439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316bf7-4e7d-4c2f-806e-a90b48bd2da8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.042934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.044235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.044601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.045063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.045988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.048035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.059837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.061984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.066312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1066/2104] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:21.071997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.082717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff913e6-dc16-4f74-a4fd-c9ef2386c976 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.090505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4998422-2a38-403b-8a43-0d4c11273cc8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.096043 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.097712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.098377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.099064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] PATH_INFO: `/v3/projects/67df224121d14a90b455960cfb919b72` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.101459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67df224121d14a90b455960cfb919b72)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.108138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.108359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.109062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.109425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.109728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] PATH_INFO: `/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07/users/c2bb07a78e374d349caf0f9089985106/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.148366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9e2afecfb0478c85c4e1c8d25ffe07, user_id=c2bb07a78e374d349caf0f9089985106, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.158668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.184045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1067/2105] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:21 2022] PUT /identity/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07/users/c2bb07a78e374d349caf0f9089985106/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:21.201294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.227457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67df224121d14a90b455960cfb919b72', 'request_id': 'req-725e34b0-366b-4dba-b1e3-7397d800df0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:21.239924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67df224121d14a90b455960cfb919b72', 'request_id': 'req-725e34b0-366b-4dba-b1e3-7397d800df0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:21.253185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67df224121d14a90b455960cfb919b72', 'request_id': 'req-725e34b0-366b-4dba-b1e3-7397d800df0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:21.289320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] The token cache is being invalidate because project 67df224121d14a90b455960cfb919b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:21.289734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:21.295396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-725e34b0-366b-4dba-b1e3-7397d800df0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:21.306737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1039/2106] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:21 2022] DELETE /identity/v3/projects/67df224121d14a90b455960cfb919b72 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:21.309015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:21.322219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd47c0f-90ee-46ad-81ea-ba3a90211bcb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.322219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd47c0f-90ee-46ad-81ea-ba3a90211bcb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.322219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd47c0f-90ee-46ad-81ea-ba3a90211bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.349942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e131f-ee38-466d-b769-0a12288d81a7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.369439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cdd47c0f-90ee-46ad-81ea-ba3a90211bcb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:21.370842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cdd47c0f-90ee-46ad-81ea-ba3a90211bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.491985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1040/2107] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:21.500719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.502994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.503930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.504212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.504500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.505248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.505915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.506191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.512615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.531438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1068/2108] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:21.535952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.538033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.549468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:21.551113 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.563338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1041/2109] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:21.570247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.570494 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a18aa-06a3-436a-b3fc-282818336e60 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.576983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.577903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.578196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.578518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] PATH_INFO: `/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07/users/c2bb07a78e374d349caf0f9089985106/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.581239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9e2afecfb0478c85c4e1c8d25ffe07, user_id=c2bb07a78e374d349caf0f9089985106, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.584025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5379ef-3082-4540-8261-cffae1b681e2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.591259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f5295-9e4c-470b-8672-e1cc2e450c3d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.610051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1069/2110] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:21 2022] PUT /identity/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07/users/c2bb07a78e374d349caf0f9089985106/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:21.629414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.629903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.630449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.676132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] MFA Rules not processed for user `c2bb07a78e374d349caf0f9089985106`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:21.677440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.709569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.710212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.710555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.710804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.711537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:21.711872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.713773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.730789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1042/2111] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:21.737084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.754483 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:21.762912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf502-c91a-4555-a08b-1175bae6ed9f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.775397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.776386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.776910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.778254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.780644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.784361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1070/2112] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:21.786756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.811058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.813447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.840175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1043/2113] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:21 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:21.842617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:21.941132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b70e-844c-4c19-ba3e-62dcac6200b0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.962946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:22.029984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a9bf3-2e27-44d3-a66f-dfc8b2c84383 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.048118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.054126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.054126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.054126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.054126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.054850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.074732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.076699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.078746 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.079483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.115096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1044/2114] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:21 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:22.117430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.120494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.176799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b983-4f4c-4a87-8440-bf631e8ea447 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.189191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.192121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.192984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.193965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] PATH_INFO: `/v3/users/b334f6d4b8cf442399062477d5d2d7cf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.196920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b334f6d4b8cf442399062477d5d2d7cf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.201984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.271921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.274124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.290677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1071/2115] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:22.291557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.292632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b334f6d4b8cf442399062477d5d2d7cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:22.304129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b334f6d4b8cf442399062477d5d2d7cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:22.306104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0d54-a59f-4727-9714-94a7e59ec1ea service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.321547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b334f6d4b8cf442399062477d5d2d7cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:22.322663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b334f6d4b8cf442399062477d5d2d7cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:22.323314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.324060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.324300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.324578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.325482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.326996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90d310e3-e269-4ac3-beca-d0dd6bfb1369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b334f6d4b8cf442399062477d5d2d7cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:22.327154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.340085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1072/2116] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:22.348646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.351836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.353631 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1045/2117] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:22 2022] DELETE /identity/v3/users/b334f6d4b8cf442399062477d5d2d7cf => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:22.358781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.359033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.359292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.375152 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbea22-935e-46db-9c74-9fb4ee1f129d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.380633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:22.381986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.433527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:22.457932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1046/2118] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:22.458848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.460244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.477046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.478133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.478772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.478772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.479485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.481517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.499445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1073/2119] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:22.500466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.502605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.546787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069636-504c-4d55-9dcd-7302bbe7d9bc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.551766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.552694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.553087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.553706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.555832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.571180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:22.580432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467fd5e-96d4-4889-8923-a91fdfc85c13 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.592545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.592545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.592545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.592545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.606590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.609322 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.621643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.623564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87f197-e100-40cb-9ed7-7c2748168037 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.630362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1074/2120] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:22.631717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.637469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1047/2121] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:22.637469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.637469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.637469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.647149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272df97-4e31-4011-859e-f9fa07e9b582 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:22.659254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:22.660378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.660947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.661231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] PATH_INFO: `/v3/projects/c673cdee091a4bd081ced1f35d04b5ba/users/0fbe1eec8c7b4ca3872203d3e2634681/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.672414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] MFA Rules not processed for user `33e56d53506d49a68c864556486c94c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:22.675090 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.680422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c673cdee091a4bd081ced1f35d04b5ba, user_id=0fbe1eec8c7b4ca3872203d3e2634681, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:22.684721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc117f-e5f3-43ce-95f8-1d8edd7324df admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:22.701872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1075/2122] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:22 2022] PUT /identity/v3/projects/c673cdee091a4bd081ced1f35d04b5ba/users/0fbe1eec8c7b4ca3872203d3e2634681/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:22.714713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:22.715151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:22.715311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:22.763260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] MFA Rules not processed for user `0fbe1eec8c7b4ca3872203d3e2634681`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:22.765017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:22.797852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1048/2123] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:22.918773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1076/2124] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:22.926309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:22.941577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44d778-ef89-403b-a974-59b8b6b7d897 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.010820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:23.038445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.038445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.039673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.039673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.039673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.041922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.082272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.083809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1049/2125] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:22 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:23.088824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.090913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:23.101179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da509-f7d7-49ac-a4aa-7a5cb0af6ab8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.190845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.191575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.191868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.192153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.192851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.196027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.196309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05ea78-c14c-4598-b2ee-8d7b53c299b3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.202047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.203031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.203385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.203756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] PATH_INFO: `/v3/projects/a8c9632f73b94362a561966e2691e7b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.205111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c9632f73b94362a561966e2691e7b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.207239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.221039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1077/2126] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:23 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:23.224474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.237027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780adc5-e525-4ca0-9927-d6274e61faf8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.247862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.249047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.249562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.250083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.251591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.254864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.276691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1078/2127] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:23.285360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.293918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c9632f73b94362a561966e2691e7b2', 'request_id': 'req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.305127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c9632f73b94362a561966e2691e7b2', 'request_id': 'req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.306722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0c2c-744a-4025-8292-293e97c384bc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.318382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c9632f73b94362a561966e2691e7b2', 'request_id': 'req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.333000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] The token cache is being invalidate because project a8c9632f73b94362a561966e2691e7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:23.333322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.334147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.336248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1050/2128] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:23 2022] DELETE /identity/v3/projects/a8c9632f73b94362a561966e2691e7b2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:23.344505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.347051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:23.387943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:23.434777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.435563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.435899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.436232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] PATH_INFO: `/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec/users/43dff8829cb942d7a06b322b96481b96/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.463945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf1c9fa835e4bc19aa7acec024ef4ec, user_id=43dff8829cb942d7a06b322b96481b96, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.468620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fb954-21f4-4ceb-9d62-65f9212ea3e0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.470869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.475165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.476078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.476553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.476831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] PATH_INFO: `/v3/users/9978dfa266584eaf8dc3336399bd7c80` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.478231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9978dfa266584eaf8dc3336399bd7c80)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.480263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.482139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:23.502162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1079/2129] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:23 2022] PUT /identity/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec/users/43dff8829cb942d7a06b322b96481b96/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:23.503892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.507400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:23.596968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9978dfa266584eaf8dc3336399bd7c80'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.616184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9978dfa266584eaf8dc3336399bd7c80'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.622064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba548-a936-4a18-9d41-125609dc75e4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.638438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.638438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.638438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.638438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.638977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.638977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.649268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9978dfa266584eaf8dc3336399bd7c80'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.650036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9978dfa266584eaf8dc3336399bd7c80'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.650705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-177f743b-7660-4a7f-971f-64e9a508faaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9978dfa266584eaf8dc3336399bd7c80'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:23.659806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1080/2130] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:23.661935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:23.663973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:23.689721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d3c49-ed2f-48ef-be14-610052b926e1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:23.690327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1051/2131] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:23 2022] DELETE /identity/v3/users/9978dfa266584eaf8dc3336399bd7c80 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:23.798433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:23.800563 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.802072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.802847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] PATH_INFO: `/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec/users/43dff8829cb942d7a06b322b96481b96/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.807987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf1c9fa835e4bc19aa7acec024ef4ec, user_id=43dff8829cb942d7a06b322b96481b96, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:23.812023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:23.829415 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1081/2132] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:23 2022] PUT /identity/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec/users/43dff8829cb942d7a06b322b96481b96/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:23.841157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:23.841390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:23.841591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:23.885883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] MFA Rules not processed for user `43dff8829cb942d7a06b322b96481b96`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:23.890328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.030920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1052/2133] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:24.044239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:24.046051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.126083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:24.136675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510a69d-bd5e-4fc3-ae1d-e2530dcf9354 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:24.141949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:24.143250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.143541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.143935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.145985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.245187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:24.247018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.274994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:24.274994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376ad2b-ae6b-4f0a-8016-3dd3b4287e59 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:24.296019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1082/2134] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:24.405917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f308d-0ebd-4060-b9c1-1d003b0babc4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:24.412446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:24.414023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.414312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.414637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] PATH_INFO: `/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.416263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b08a0d8798e4dfbbe6ad25802ce0d96)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:24.419139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:24.433218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:24.508620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b08a0d8798e4dfbbe6ad25802ce0d96', 'request_id': 'req-e771c640-697c-473a-a501-2db7e4eb1ec9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:24.526321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b08a0d8798e4dfbbe6ad25802ce0d96', 'request_id': 'req-e771c640-697c-473a-a501-2db7e4eb1ec9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:24.536284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b08a0d8798e4dfbbe6ad25802ce0d96', 'request_id': 'req-e771c640-697c-473a-a501-2db7e4eb1ec9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:24.565153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] The token cache is being invalidate because project 3b08a0d8798e4dfbbe6ad25802ce0d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:24.565153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:24.565153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e771c640-697c-473a-a501-2db7e4eb1ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:24.571353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1053/2135] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:24 2022] DELETE /identity/v3/projects/3b08a0d8798e4dfbbe6ad25802ce0d96 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:24.579285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.579939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.580668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.693350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:24.694980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.870536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1083/2136] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:24.874776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.875072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.875426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.908799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:24.911405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.932098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1054/2137] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:24.936051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:24.956692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1b5d-a9d8-4975-bc5c-2fe07e808dc4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:24.964077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:24.965937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.966444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.966975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.967829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:24.968944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:24.970116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:24.990452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1084/2138] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:24.994856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.009985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b69fd-3092-4962-8735-2c8e45ca036d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.013324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.014251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.014665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.014845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.015566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.021808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.066461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1055/2139] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:24 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:25.070543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.089618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a78aea-b620-4660-8705-11b135b05bd7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.174878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.175356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.175698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.176163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.177195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.180694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.213780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1085/2140] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:25 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:25.215197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.231281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd94829-f956-46b0-ac3e-ebaa18e51053 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.240203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.241377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.241882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.242380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.243825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.247073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.271477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1056/2141] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:25.273578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.291655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecf4bc-d0da-4ace-837b-7c469f165053 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.298511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.299242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.299534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.299816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.300566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.302239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef07cd-fc78-4e2b-8470-cf5113a89e3d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.322509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1086/2142] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:25.327076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.341344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58880e9-d2d4-4d3f-a5de-da5db524f370 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.349955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.350975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.351258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.351532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] PATH_INFO: `/v3/projects/72b8c005ec974c09a0157a38bec623e1/users/27fa3301b7aa4b6ba0298408030b9660/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.368540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b8c005ec974c09a0157a38bec623e1, user_id=27fa3301b7aa4b6ba0298408030b9660, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.371350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.386435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1057/2143] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:25 2022] PUT /identity/v3/projects/72b8c005ec974c09a0157a38bec623e1/users/27fa3301b7aa4b6ba0298408030b9660/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:25.399558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.399825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.400109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.413548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:25.436599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] MFA Rules not processed for user `27fa3301b7aa4b6ba0298408030b9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:25.438190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:25.534246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1087/2144] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:25.541795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.662017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7223527-161a-4c8e-b934-7c7c7e28449d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.670692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.671523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.671931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.672236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.673282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.676352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.716110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1058/2145] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:25 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:25.721846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.749950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45bf48-98f6-421f-946b-fda9f719bfb7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.839320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.840031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.840339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.840614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.841310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.843675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.877001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1088/2146] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:25 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:25.879493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.896168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588de6-2578-430a-a03c-0d20b034987b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.902149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.902787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.903068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.903343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.904110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.906696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:25.920224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1059/2147] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:25.925775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:25.940892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d799a3b-6455-4559-9023-b771af23df67 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:25.947644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:25.948789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:25.949210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:25.949560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] PATH_INFO: `/v3/projects/6be7b13f324d47f1b2b40e12bafec089/users/8631ad726be54c309882c01fa4b0d482/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:25.988346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be7b13f324d47f1b2b40e12bafec089, user_id=8631ad726be54c309882c01fa4b0d482, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:25.994166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:26.028521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1089/2148] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:25 2022] PUT /identity/v3/projects/6be7b13f324d47f1b2b40e12bafec089/users/8631ad726be54c309882c01fa4b0d482/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:26.032467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:26.041186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:26.048067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2f1a5-9cfb-4d59-b1fe-0fdd18803a03 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:26.162776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:26.164124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:26.164124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:26.164124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:26.164740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:26.166492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e152fe7-bd3a-4191-8274-13d48312a564 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:26.179163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1060/2149] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:26.184247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:26.200383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecdd17-aa68-4ed3-8a8f-1531d867562a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:26.206968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:26.208285 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:26.209219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:26.209506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] PATH_INFO: `/v3/projects/6be7b13f324d47f1b2b40e12bafec089/users/8631ad726be54c309882c01fa4b0d482/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:26.212289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be7b13f324d47f1b2b40e12bafec089, user_id=8631ad726be54c309882c01fa4b0d482, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:26.215220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:26.233302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1090/2150] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:26 2022] PUT /identity/v3/projects/6be7b13f324d47f1b2b40e12bafec089/users/8631ad726be54c309882c01fa4b0d482/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:26.248472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:26.249062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:26.249062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:26.282715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] MFA Rules not processed for user `8631ad726be54c309882c01fa4b0d482`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:26.284521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:26.409707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1061/2151] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:26.424144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:26.427440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:26.539914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:26.548035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838f8a-ab54-4853-90d3-e5660bcf447b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:26.558162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:26.559154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:26.559541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:26.559924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:26.577594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:26.579287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:26.591999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1091/2152] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:26.742546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:26.754033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6c7e-f7d7-484a-bbfc-2bb061147873 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:26.761098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:26.761773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:26.762156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:26.762464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:26.842358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:26.843891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1446-60b4-4d3e-94f4-33d40023a14f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:26.855025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1062/2153] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:27.172026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:27.245760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e72216-ae82-43fe-a11d-000b7e47c5cf service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:27.252026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:27.252920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.253207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.253557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] PATH_INFO: `/v3/users/27fa3301b7aa4b6ba0298408030b9660` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.254801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fa3301b7aa4b6ba0298408030b9660)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:27.256782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:27.331767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fa3301b7aa4b6ba0298408030b9660'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:27.339647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fa3301b7aa4b6ba0298408030b9660'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:27.357831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fa3301b7aa4b6ba0298408030b9660'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:27.369710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fa3301b7aa4b6ba0298408030b9660'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:27.370658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fa3301b7aa4b6ba0298408030b9660'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:27.380845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1092/2154] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:27 2022] DELETE /identity/v3/users/27fa3301b7aa4b6ba0298408030b9660 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:27.387070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.387695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.387872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.412656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:27.414364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.441373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:27.499012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1063/2155] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:27.509308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:27.510862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.570936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36821a-4d50-4304-b53b-29cd0064495d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:27.574561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:27.575212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.575498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.575772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.585795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:27.587215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:27.597911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1093/2156] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:27.945021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:27.946774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.966192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316cae-c5cb-4afd-ad3e-0ddb48d5b950 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:27.971552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:27.972437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.972713 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.973003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] PATH_INFO: `/v3/projects/72b8c005ec974c09a0157a38bec623e1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.974251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b8c005ec974c09a0157a38bec623e1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:27.976239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:28.027441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b8c005ec974c09a0157a38bec623e1', 'request_id': 'req-d35b935f-719e-45b7-b75a-14f4206ff35d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.039078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b8c005ec974c09a0157a38bec623e1', 'request_id': 'req-d35b935f-719e-45b7-b75a-14f4206ff35d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.046637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b8c005ec974c09a0157a38bec623e1', 'request_id': 'req-d35b935f-719e-45b7-b75a-14f4206ff35d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.058347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] The token cache is being invalidate because project 72b8c005ec974c09a0157a38bec623e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:28.058654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.059333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.060896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1064/2157] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:27 2022] DELETE /identity/v3/projects/72b8c005ec974c09a0157a38bec623e1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:28.065906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:28.076212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d46ddb-b4b8-4b48-a68a-43ae474d4ae5 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:28.086627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:28.139028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:28.140084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:28.140255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:28.140424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] PATH_INFO: `/v3/users/8631ad726be54c309882c01fa4b0d482` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:28.141730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8631ad726be54c309882c01fa4b0d482)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:28.144399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:28.209899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8631ad726be54c309882c01fa4b0d482'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.228123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8631ad726be54c309882c01fa4b0d482'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.243553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8631ad726be54c309882c01fa4b0d482'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.253632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8631ad726be54c309882c01fa4b0d482'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.255092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee53f37-c966-42f0-b33d-81a088e924f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8631ad726be54c309882c01fa4b0d482'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.265051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1094/2158] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:28 2022] DELETE /identity/v3/users/8631ad726be54c309882c01fa4b0d482 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:28.617932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:28.619719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:28.640905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b935f-719e-45b7-b75a-14f4206ff35d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:28.716460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:28.717352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:28.717677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:28.717954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] PATH_INFO: `/v3/projects/6be7b13f324d47f1b2b40e12bafec089` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:28.719328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be7b13f324d47f1b2b40e12bafec089)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:28.721630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:28.780648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be7b13f324d47f1b2b40e12bafec089', 'request_id': 'req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.782743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:28.790712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be7b13f324d47f1b2b40e12bafec089', 'request_id': 'req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.799877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be7b13f324d47f1b2b40e12bafec089', 'request_id': 'req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.817207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] The token cache is being invalidate because project 6be7b13f324d47f1b2b40e12bafec089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:28.817708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.818765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9f1f75d6-1a2c-4393-9b22-d3a6a3fcb84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:28.821316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1065/2159] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:28 2022] DELETE /identity/v3/projects/6be7b13f324d47f1b2b40e12bafec089 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:28.828172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:28.828488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:28.828756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:28.854948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:28.857042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:28.936764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1095/2160] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:28.946008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:28.946305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:28.946619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:28.966852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:28.968085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:28.976856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1066/2161] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:28.981577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:28.998807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ff84-ebd7-4b64-bf38-b24463fe1ecb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.004341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.005054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.005440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.005602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.009568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:29.009568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.009568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.025291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1096/2162] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:29.026943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.041216 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a42b0-7163-430a-a487-61e6b027c5ae None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.050725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.051838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.052313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.052782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.054188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.057952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.089716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1067/2163] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:29 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:29.094220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.105014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224880-fb0e-4a57-9a92-365e09d85001 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.123313 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:29.167918 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.168601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.168888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.169174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.169895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.171906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.199352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1097/2164] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:29 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:29.204248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.217076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd376e-474a-459f-b236-b9d15f6c65cb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.222116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.222940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.223224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.223498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.224241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.225872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.246011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1068/2165] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:29.252318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.270176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8025-ffd1-465f-9ea4-467b5995b893 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.277780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.278921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.279429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.279940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.281281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.284510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82e2bf-1909-4fc2-bb2b-dc402c26b49a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.300538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1098/2166] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:29.306086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.323987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea7e40-8151-4c2f-ac46-79a85d9985be admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.331725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.333264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.333754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.334254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] PATH_INFO: `/v3/projects/f76540bb35d94b1eb9399317e8c172fb/users/fbe57f4cfe0449b59637d30b925acf48/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.360485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76540bb35d94b1eb9399317e8c172fb, user_id=fbe57f4cfe0449b59637d30b925acf48, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.366358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.384798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1069/2167] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:29 2022] PUT /identity/v3/projects/f76540bb35d94b1eb9399317e8c172fb/users/fbe57f4cfe0449b59637d30b925acf48/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:29.397686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.398229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.398761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.434092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] MFA Rules not processed for user `fbe57f4cfe0449b59637d30b925acf48`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:29.435683 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:29.537087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1099/2168] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:29.549445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:29.552566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:29.636682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f200-2634-45be-8930-e17c4cdce49a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:29.640817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:29.641490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.641778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.642086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.665779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:29.668914 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:29.686859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1070/2169] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:31.670961 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:31.673977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:31.691129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0e84e-b431-41c0-8f8f-a523efcfe416 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:31.706870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:31.707694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:31.708134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:31.708298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:31.711629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:31.797002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:31.799533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cd7f1-906e-4026-a285-e3c91f15c312 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:31.821920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1100/2170] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:34.346884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:34.426031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee04d-2d93-47ba-adc0-a1075ebb8a34 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:34.433745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:34.434680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:34.434961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:34.435241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] PATH_INFO: `/v3/users/fbe57f4cfe0449b59637d30b925acf48` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:34.436663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe57f4cfe0449b59637d30b925acf48)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:34.439312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:34.464305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=3,keystone:DELETE=5,keystone:SELECT=92 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:34.505938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe57f4cfe0449b59637d30b925acf48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:34.519480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe57f4cfe0449b59637d30b925acf48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:34.538509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe57f4cfe0449b59637d30b925acf48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:34.541153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe57f4cfe0449b59637d30b925acf48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:34.542290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe57f4cfe0449b59637d30b925acf48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:34.562630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1071/2171] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:34 2022] DELETE /identity/v3/users/fbe57f4cfe0449b59637d30b925acf48 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:34.567882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b024-87df-4705-9988-9913de511240 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:34.568669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b024-87df-4705-9988-9913de511240 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:34.568669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b024-87df-4705-9988-9913de511240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:34.587875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-77a4b024-87df-4705-9988-9913de511240 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:34.589676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-77a4b024-87df-4705-9988-9913de511240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:34.627828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:34.716873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1101/2172] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:34.736791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:34.741143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:34.944907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c30df-1e82-428f-96c9-e35250233221 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:34.964795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:34.966857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:34.967383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:34.967762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:35.005293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:35.016188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886b5b9-eb5f-4469-9c6b-d149ad058f61 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:35.056767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1072/2173] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:35.692851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4b024-87df-4705-9988-9913de511240 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:35.694439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-77a4b024-87df-4705-9988-9913de511240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:35.707733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4b024-87df-4705-9988-9913de511240 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:35.714723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:35.716105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:35.716606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:35.717044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] PATH_INFO: `/v3/projects/f76540bb35d94b1eb9399317e8c172fb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:35.718935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76540bb35d94b1eb9399317e8c172fb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:35.722760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:35.818236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76540bb35d94b1eb9399317e8c172fb', 'request_id': 'req-c671a06e-e377-423e-9cd8-767c35cc7cd2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:35.834105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76540bb35d94b1eb9399317e8c172fb', 'request_id': 'req-c671a06e-e377-423e-9cd8-767c35cc7cd2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:35.849401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76540bb35d94b1eb9399317e8c172fb', 'request_id': 'req-c671a06e-e377-423e-9cd8-767c35cc7cd2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:35.882822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] The token cache is being invalidate because project f76540bb35d94b1eb9399317e8c172fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:35.883725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:35.885625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c671a06e-e377-423e-9cd8-767c35cc7cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:35.890158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1102/2174] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:35 2022] DELETE /identity/v3/projects/f76540bb35d94b1eb9399317e8c172fb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:35.897702 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:35.898508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:35.898508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:35.939259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:35.943076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.071139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1073/2175] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:36.080977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.081502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.082001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.100921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.105530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.127307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:36.129989 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.135003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7ea8f-ce97-44ae-8ea3-e4242c239fa6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.142959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.145106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.145602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.146392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] PATH_INFO: `/v3/users/2acccf73077e43ca84171315cc463acc` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.148419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2acccf73077e43ca84171315cc463acc)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.151953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.154966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1103/2176] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:36.162100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.184866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739cfb0-f65c-47c0-88f0-3746765433f1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.200891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.202172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.202741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.203783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.205745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:36.206337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.208403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.234922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1104/2177] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:36.243577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.273085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8522-36a8-40f8-a31c-0b2a6659e90b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.276077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:36.292349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.293852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.294786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.295745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.298001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.303464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.303679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2acccf73077e43ca84171315cc463acc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:36.320017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2acccf73077e43ca84171315cc463acc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:36.383063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1105/2178] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:36 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:36.390992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.395582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2acccf73077e43ca84171315cc463acc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:36.398233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2acccf73077e43ca84171315cc463acc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:36.399207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3c7b7825-5db2-4edf-a463-a861eef783f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2acccf73077e43ca84171315cc463acc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:36.455784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1074/2179] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:36 2022] DELETE /identity/v3/users/2acccf73077e43ca84171315cc463acc => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:36.470822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.471355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.471785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.498801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:36.500023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.577409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae580f9d-2300-4fbd-8c97-5c00bdf9c00b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.624407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:36.647481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1075/2180] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:36.664845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.668676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.753652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.755466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.755923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.756420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.759454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.770803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.818190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbe51c-3187-42ac-b1fc-816752d6b10b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.822073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1106/2181] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:36 2022] POST /identity/v3/users => generated 295 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:36.827138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.827366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.828295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.828915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.829577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.829798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.850316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.853608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.868616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734ed61-d8bf-42de-bbab-1e0750c8cd74 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.875739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1076/2182] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:36.892612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.894005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.895738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.896589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.898595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.903062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.938512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1107/2183] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:36.940675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:36.957877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f7e1-ac68-465c-90a1-cf01d4f9528b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.967545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.968530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.969585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.970670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.972051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.974755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e23c7-3075-4fa6-accd-23eab711360b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.995111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1077/2184] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:36.998048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:37.016781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e462-7744-47c0-9479-da0e2c690b8a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:37.029075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:37.034121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:37.036392 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:37.036930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] PATH_INFO: `/v3/projects/aa3853573e1747c18d79bb90e297281b/users/809e4d438c9747698bcf0f16af286879/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:37.109770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3853573e1747c18d79bb90e297281b, user_id=809e4d438c9747698bcf0f16af286879, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:37.115474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:37.151891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1108/2185] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:36 2022] PUT /identity/v3/projects/aa3853573e1747c18d79bb90e297281b/users/809e4d438c9747698bcf0f16af286879/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:37.172686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:37.174180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:37.174688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:37.229294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] MFA Rules not processed for user `809e4d438c9747698bcf0f16af286879`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:37.231610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:37.315620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:37.403560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1078/2186] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:37.437733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:37.439475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:37.499664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:37.503168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:37.556286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba399f-84fb-4c7d-a08c-8d54952f567b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:37.561616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:37.562194 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:37.562529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:37.562813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:37.576547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:37.578364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4335-5b0d-4958-9bce-0d9759f906ae service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:37.593820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1109/2187] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:37.684878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524925b6-f31e-4a25-86cb-f3dc362654af None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:37.696876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:37.697869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:37.698229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:37.698626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] PATH_INFO: `/v3/projects/7291137bcc13437884d14bc24d1f67a4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:37.700457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7291137bcc13437884d14bc24d1f67a4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:37.702855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:37.773834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884984-73a7-4114-991b-271499952c1a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:37.774651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884984-73a7-4114-991b-271499952c1a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:37.774651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884984-73a7-4114-991b-271499952c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:37.810756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-df884984-73a7-4114-991b-271499952c1a None None] MFA Rules not processed for user `ab3be59ea4c74da3b4e19273102bccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:37.818503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-df884984-73a7-4114-991b-271499952c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:37.827658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7291137bcc13437884d14bc24d1f67a4', 'request_id': 'req-87f23f56-a1c6-470c-824b-af8b6cac0868'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:37.847642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7291137bcc13437884d14bc24d1f67a4', 'request_id': 'req-87f23f56-a1c6-470c-824b-af8b6cac0868'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:37.870910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7291137bcc13437884d14bc24d1f67a4', 'request_id': 'req-87f23f56-a1c6-470c-824b-af8b6cac0868'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:37.903224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] The token cache is being invalidate because project 7291137bcc13437884d14bc24d1f67a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:37.903548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:37.904559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:37.913128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1079/2188] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:37 2022] DELETE /identity/v3/projects/7291137bcc13437884d14bc24d1f67a4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:37.914125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:37.917995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:37.928424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1110/2189] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:38.024729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23f56-a1c6-470c-824b-af8b6cac0868 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:38.035103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:38.037251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:38.037867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:38.038566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] PATH_INFO: `/v3/users/c2bb07a78e374d349caf0f9089985106` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:38.041850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bb07a78e374d349caf0f9089985106)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:38.046661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:38.073598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:38.162239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bb07a78e374d349caf0f9089985106'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.177688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bb07a78e374d349caf0f9089985106'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.227122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bb07a78e374d349caf0f9089985106'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.228027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bb07a78e374d349caf0f9089985106'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.228803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bb07a78e374d349caf0f9089985106'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.256715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1080/2190] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:37 2022] DELETE /identity/v3/users/c2bb07a78e374d349caf0f9089985106 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:38.508278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df884984-73a7-4114-991b-271499952c1a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:38.509825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-df884984-73a7-4114-991b-271499952c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:38.537829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df884984-73a7-4114-991b-271499952c1a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:38.561233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:38.683826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:38.684931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:38.685558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:38.685908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] PATH_INFO: `/v3/users/0fbe1eec8c7b4ca3872203d3e2634681` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:38.690103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbe1eec8c7b4ca3872203d3e2634681)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:38.692425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:38.764102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:38.769522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:38.807172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbe1eec8c7b4ca3872203d3e2634681'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.819200 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbe1eec8c7b4ca3872203d3e2634681'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.830884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667800-b1b8-492e-8d2b-936cc5230112 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:38.834531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbe1eec8c7b4ca3872203d3e2634681'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.846627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbe1eec8c7b4ca3872203d3e2634681'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.847445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3143019f-5623-485c-978b-df36472a4160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbe1eec8c7b4ca3872203d3e2634681'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:38.848335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:38.849884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:38.850506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:38.851054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] PATH_INFO: `/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:38.853435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9e2afecfb0478c85c4e1c8d25ffe07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:38.857245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1111/2191] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:38 2022] DELETE /identity/v3/users/0fbe1eec8c7b4ca3872203d3e2634681 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:38.862624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:38.863888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:38.864443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:38.864999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:38.890303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:38.891696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.000286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1112/2192] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:39.002866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9e2afecfb0478c85c4e1c8d25ffe07', 'request_id': 'req-537209f4-3216-4590-957b-4ec66b698d2e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:39.013188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:39.017080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.018526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9e2afecfb0478c85c4e1c8d25ffe07', 'request_id': 'req-537209f4-3216-4590-957b-4ec66b698d2e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:39.036537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9e2afecfb0478c85c4e1c8d25ffe07', 'request_id': 'req-537209f4-3216-4590-957b-4ec66b698d2e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:39.062977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] The token cache is being invalidate because project 4a9e2afecfb0478c85c4e1c8d25ffe07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:39.063499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:39.064737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-537209f4-3216-4590-957b-4ec66b698d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:39.068884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1081/2193] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:38 2022] DELETE /identity/v3/projects/4a9e2afecfb0478c85c4e1c8d25ffe07 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:39.076326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3f90-a145-4619-98e3-d9f33800cb07 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.076977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3f90-a145-4619-98e3-d9f33800cb07 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.077909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a3f90-a145-4619-98e3-d9f33800cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.108537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:39.118910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b13a3f90-a145-4619-98e3-d9f33800cb07 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:39.121261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b13a3f90-a145-4619-98e3-d9f33800cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.157515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3895cc0-305d-4dad-888a-2f0e97e5b628 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:39.234028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1082/2194] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:39.255565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.256037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.256537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.278618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:39.281654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:39.283431 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.284266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.285055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.288769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.288954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:39.310079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.344998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:39.347119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:39.370772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1083/2195] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:39.381131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:39.418010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1113/2196] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:39.430904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30affc3e-498f-4d36-ac59-3b0b60078d11 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:39.449089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:39.450687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.451413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.452186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.453853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:39.455309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:39.461089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:39.487290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1084/2197] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:39.494841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:39.568674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad14ed9-d199-44fd-a982-a24964d47d06 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:39.602668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:39.605051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.605628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.606179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.608144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:39.613658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:39.705026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1114/2198] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:39 2022] POST /identity/v3/projects => generated 383 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:39.712872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:39.744865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856c60-b535-4a53-97f2-ab9c9f5bcf2d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:39.991724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:39.991724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.992517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.993100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.994534 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:39.999800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.051264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1085/2199] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:39 2022] POST /identity/v3/users => generated 292 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:40.056717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.091478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038229-2548-449d-ae5b-3a3ee7965d4c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.103579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.104478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.104647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.104892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.105633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.107542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.122917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.125070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:40.135551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1115/2200] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:40.141141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.143738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deed9e-3ae9-4bab-8128-94b1a333667f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.149731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.150673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.151055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.151393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] PATH_INFO: `/v3/projects/c673cdee091a4bd081ced1f35d04b5ba` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.152811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c673cdee091a4bd081ced1f35d04b5ba)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.154971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.170462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0c188-dbab-4318-98b0-298a6cc95ca9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.182147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.183367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.183916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.184420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.186088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.190063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.216481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1116/2201] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:40.223817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.241072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6757246-3908-47f6-ba3e-8af2babade6b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.271607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c673cdee091a4bd081ced1f35d04b5ba', 'request_id': 'req-0876c6ca-9207-4e39-b4ee-17deae78020d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.284163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c673cdee091a4bd081ced1f35d04b5ba', 'request_id': 'req-0876c6ca-9207-4e39-b4ee-17deae78020d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.300319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c673cdee091a4bd081ced1f35d04b5ba', 'request_id': 'req-0876c6ca-9207-4e39-b4ee-17deae78020d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.318903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] The token cache is being invalidate because project c673cdee091a4bd081ced1f35d04b5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:40.320203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.321236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.323865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1086/2202] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:40 2022] DELETE /identity/v3/projects/c673cdee091a4bd081ced1f35d04b5ba => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:40.330438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.330675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.331619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.331892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.332179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] PATH_INFO: `/v3/projects/c07ae9d0b43e4ff9a13b61e5654b0ec0/users/0321034e38334c6e8909179ca67349dc/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.333219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:40.350235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07ae9d0b43e4ff9a13b61e5654b0ec0, user_id=0321034e38334c6e8909179ca67349dc, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.350650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c6ca-9207-4e39-b4ee-17deae78020d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.353052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec70e-cfbc-4ee1-a260-5dde70709367 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.355630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.356543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.356843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.357125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] PATH_INFO: `/v3/users/43dff8829cb942d7a06b322b96481b96` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.358443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dff8829cb942d7a06b322b96481b96)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.360566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.374049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1117/2203] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:40 2022] PUT /identity/v3/projects/c07ae9d0b43e4ff9a13b61e5654b0ec0/users/0321034e38334c6e8909179ca67349dc/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:40.387536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.387909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.388107 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.420544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] MFA Rules not processed for user `0321034e38334c6e8909179ca67349dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:40.421852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:40.439980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dff8829cb942d7a06b322b96481b96'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.453333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:40.458937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:40.461566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dff8829cb942d7a06b322b96481b96'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.485384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dff8829cb942d7a06b322b96481b96'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.486877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dff8829cb942d7a06b322b96481b96'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.487895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dff8829cb942d7a06b322b96481b96'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:40.522073 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1087/2204] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:40 2022] DELETE /identity/v3/users/43dff8829cb942d7a06b322b96481b96 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:40.532676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1118/2205] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:40.538093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.540974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:40.644688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd650f-f817-47b5-9a1b-3e4b1628ce34 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.654869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.656271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.656575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.657027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.658084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.661329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.703201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1088/2206] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:40 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:40.707923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.861881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f36fb-a7d8-4dda-8092-ec0ef885a34f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.962560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:40.965123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:40.968150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.968889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.969190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.969447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:40.970273 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:40.973325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:40.989820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0f584-1bad-4419-9ac4-4178324bc4a7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:40.997557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:40.998976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:40.999454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:40.999939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] PATH_INFO: `/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.002657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf1c9fa835e4bc19aa7acec024ef4ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.003272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1119/2207] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:40 2022] POST /identity/v3/users => generated 300 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:41.006869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.009443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.032357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4090d9-7899-4901-8f5a-0f69a16968aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.039789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.040693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.041112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.041785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.043672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.046010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.063908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1120/2208] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:41.069050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.086734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1255779-6fb5-4976-800d-3af2d0dce878 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.090202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf1c9fa835e4bc19aa7acec024ef4ec', 'request_id': 'req-f8555c28-1b8f-4914-a544-5ad45b3e90f7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:41.101217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf1c9fa835e4bc19aa7acec024ef4ec', 'request_id': 'req-f8555c28-1b8f-4914-a544-5ad45b3e90f7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:41.118082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf1c9fa835e4bc19aa7acec024ef4ec', 'request_id': 'req-f8555c28-1b8f-4914-a544-5ad45b3e90f7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:41.133637 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] The token cache is being invalidate because project 1cf1c9fa835e4bc19aa7acec024ef4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:41.134612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:41.135728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f8555c28-1b8f-4914-a544-5ad45b3e90f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:41.138595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1089/2209] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:40 2022] DELETE /identity/v3/projects/1cf1c9fa835e4bc19aa7acec024ef4ec => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:41.145479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014bac-1e24-4009-bcb6-c56d51865e11 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.145903 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014bac-1e24-4009-bcb6-c56d51865e11 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.146297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014bac-1e24-4009-bcb6-c56d51865e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.162227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:41.166133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fe014bac-1e24-4009-bcb6-c56d51865e11 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:41.167763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe014bac-1e24-4009-bcb6-c56d51865e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:41.195142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.196287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.196634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.197068 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] PATH_INFO: `/v3/projects/380ec613d78e44c9a77f0c2c93847840/users/49807951096c4c6689207934b51a479a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.218834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:41.233793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=380ec613d78e44c9a77f0c2c93847840, user_id=49807951096c4c6689207934b51a479a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.240726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.256077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1090/2210] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:41.266053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.266989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.266989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.269405 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1121/2211] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:41 2022] PUT /identity/v3/projects/380ec613d78e44c9a77f0c2c93847840/users/49807951096c4c6689207934b51a479a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:41.274835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.277369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:41.297988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:41.299813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:41.372420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9369-9939-4f22-aceb-f3c5c568f8fc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.377748 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.378457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.378751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.379033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.379782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.381406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.394546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1122/2212] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:41.397642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1091/2213] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:41.399150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.402441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.410366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450621e9-b315-44d4-9e20-56795acd1f20 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.413077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04210031-44dd-4466-a500-bde6dd49bc07 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.415538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.416477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.416765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.417146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] PATH_INFO: `/v3/projects/380ec613d78e44c9a77f0c2c93847840/users/49807951096c4c6689207934b51a479a/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.418535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.419148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=380ec613d78e44c9a77f0c2c93847840, user_id=49807951096c4c6689207934b51a479a, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.419308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.419469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.419747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.420493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:41.420804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.421850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.422487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30751e1e-b72b-4b4f-8c3b-e7f3efca543f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.435157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1123/2214] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:41 2022] PUT /identity/v3/projects/380ec613d78e44c9a77f0c2c93847840/users/49807951096c4c6689207934b51a479a/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:41.436899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1092/2215] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:41.443893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.447662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.448168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.448673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.479874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] MFA Rules not processed for user `49807951096c4c6689207934b51a479a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:41.481109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:41.534128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda7e65-f23a-4dc0-8f69-4d5865eaef7c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.539221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.539899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.540180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.540453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.541173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.543159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.581147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1124/2216] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:41.581778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1093/2217] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:41.585629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.595083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.597711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:41.643146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:41.677250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6286d-8fb9-4f07-92b6-9059b95b0bae admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.685030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.686118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.686609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.687047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.688985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.690877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0430a-6c35-44b7-bf3f-489d6120544b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.692499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.698775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:41.722489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1125/2218] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:41.727424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.744011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e381f5-0f77-4995-80e2-6aae798876ec admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.753874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.754966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.755438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.755920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.757256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.760512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.781184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1126/2219] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:41.782481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.793231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc428006-273a-4b15-a2d9-8284e300890b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.793804 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.794523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.794880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.795088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.799582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.800798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.801396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.801745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.802601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.804548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.833361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1127/2220] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:41:41.837635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.851858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a857c-e33b-4d6a-99b4-a0ec2ccd625d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:41.860846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:41.861766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.862050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.862329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] PATH_INFO: `/v3/projects/4eb47c73399f4267962d7bdbbe95d19e/users/05eeb282e5de45568d06ab6b135fe343/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.883316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb47c73399f4267962d7bdbbe95d19e, user_id=05eeb282e5de45568d06ab6b135fe343, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.885079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:41.886602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b0d5-50c6-4be0-9c45-7b4e3de4e427 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.889216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:41.901025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1094/2221] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:41.910312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1128/2222] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:41 2022] PUT /identity/v3/projects/4eb47c73399f4267962d7bdbbe95d19e/users/05eeb282e5de45568d06ab6b135fe343/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:41.921743 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:41.922041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:41.922314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:41.935875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:41.959925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] MFA Rules not processed for user `05eeb282e5de45568d06ab6b135fe343`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:41.961268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:42.027685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd30a-702a-40f0-a457-c6ef395e17aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:42.035571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:42.037076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:42.037555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:42.038029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] PATH_INFO: `/v3/users/0321034e38334c6e8909179ca67349dc` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:42.040146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321034e38334c6e8909179ca67349dc)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:42.043756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:42.049391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1095/2223] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:42.065832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:42.067383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:42.115909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321034e38334c6e8909179ca67349dc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:42.130839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321034e38334c6e8909179ca67349dc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:42.141034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:42.149927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eab4c-ccbf-433a-a2f0-59a42951cd55 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:42.156881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:42.157713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321034e38334c6e8909179ca67349dc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:42.158586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:42.159136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:42.159544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:42.170665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321034e38334c6e8909179ca67349dc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:42.171392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:42.171862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83ffba08-6f7b-47f7-82e2-0d0ca55c8376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321034e38334c6e8909179ca67349dc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:42.173569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:42.175504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:42.185157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1129/2224] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:41 2022] DELETE /identity/v3/users/0321034e38334c6e8909179ca67349dc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:42.186767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1096/2225] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:42.186975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:42.187236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:42.187592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:42.214244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:42.215970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:42.308817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1130/2226] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:42.319114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:42.320575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:42.413054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba32ec-1e0f-44b5-825e-58936b7439a2 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:42.504005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:42.504976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:42.505614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:42.505967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:42.520936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:42.522797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:42.536458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1097/2227] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:42.960773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:42.962348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:42.976312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a451f-098f-4348-b826-68f74b54ac6d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:42.985128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:42.987423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:42.987721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:42.988004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] PATH_INFO: `/v3/projects/c07ae9d0b43e4ff9a13b61e5654b0ec0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:42.989545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07ae9d0b43e4ff9a13b61e5654b0ec0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:42.991551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:43.085209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07ae9d0b43e4ff9a13b61e5654b0ec0', 'request_id': 'req-def5ea0d-baf1-4325-a411-1b434795f140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.099865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07ae9d0b43e4ff9a13b61e5654b0ec0', 'request_id': 'req-def5ea0d-baf1-4325-a411-1b434795f140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.109453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07ae9d0b43e4ff9a13b61e5654b0ec0', 'request_id': 'req-def5ea0d-baf1-4325-a411-1b434795f140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.130857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] The token cache is being invalidate because project c07ae9d0b43e4ff9a13b61e5654b0ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:43.131546 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.132511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.135457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1131/2228] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:42 2022] DELETE /identity/v3/projects/c07ae9d0b43e4ff9a13b61e5654b0ec0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:43.141129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:43.158399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049a8ee-2d78-4720-94f2-0846bcdf46af service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:43.236203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:43.237594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:43.238048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:43.238595 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] PATH_INFO: `/v3/users/49807951096c4c6689207934b51a479a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:43.240577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=49807951096c4c6689207934b51a479a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:43.243298 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:43.299976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49807951096c4c6689207934b51a479a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.305390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:43.309207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49807951096c4c6689207934b51a479a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.329379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49807951096c4c6689207934b51a479a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.330138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49807951096c4c6689207934b51a479a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.330827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-dcb31473-1d98-4dd6-b0d6-44926b97b9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49807951096c4c6689207934b51a479a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:43.348310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1098/2229] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:43 2022] DELETE /identity/v3/users/49807951096c4c6689207934b51a479a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:43.779527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:43.782396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:43.801543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5ea0d-baf1-4325-a411-1b434795f140 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:43.926557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:43.927435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:43.927718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:43.928267 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] PATH_INFO: `/v3/projects/380ec613d78e44c9a77f0c2c93847840` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:43.929498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380ec613d78e44c9a77f0c2c93847840)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:43.931526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:43.966561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:43.994927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380ec613d78e44c9a77f0c2c93847840', 'request_id': 'req-8c55b898-995c-4e4a-8db7-6d6082038635'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:44.010025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380ec613d78e44c9a77f0c2c93847840', 'request_id': 'req-8c55b898-995c-4e4a-8db7-6d6082038635'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:44.029058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380ec613d78e44c9a77f0c2c93847840', 'request_id': 'req-8c55b898-995c-4e4a-8db7-6d6082038635'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:44.058361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] The token cache is being invalidate because project 380ec613d78e44c9a77f0c2c93847840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:44.058716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:44.059481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8c55b898-995c-4e4a-8db7-6d6082038635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:44.064749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1132/2230] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:43 2022] DELETE /identity/v3/projects/380ec613d78e44c9a77f0c2c93847840 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:44.068941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.069419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.069882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.120993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:44.122936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.241441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1099/2231] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:44.244305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.244884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.245242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.268869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:44.270343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.279735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1133/2232] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:44.287379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.303518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f74a-fa75-4800-b311-b4cdbd29c07e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.311818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.312620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.312991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.313360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.314189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:44.314618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.316548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.338504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1100/2233] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:44.339881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.351949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e673d-eed7-4664-97fc-11e66b302784 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.365379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.365379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.365379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.365379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.365898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.365898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.436896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1134/2234] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/projects => generated 407 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:44.442290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.455856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44163-5ad5-43dc-bdbe-54fec6855e04 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.557304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.557944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.558368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.558703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.559417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.562673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.603035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1101/2235] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/users => generated 304 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:44.608850 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.626572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410fc4-ed5f-4f13-8be8-239ee9b802df admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.643630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.644689 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.645196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.645535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.646366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.649232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.669835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1135/2236] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:44.675558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.695720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43046a6-a1e7-4f57-8300-40b6fa2824cf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.706234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.706883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.707167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.707467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.708248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.710740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b08e-1140-4955-b9b0-92a40f95f8bf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.722603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1102/2237] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:44.728493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.747303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b44-5c91-4453-9a0c-9c01056889a3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.755901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.757508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.757931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.758383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] PATH_INFO: `/v3/projects/e9cc4a806ada4382a750b162ea41a359/users/19395692e62a407fa84b1449d299e1f4/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.781110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cc4a806ada4382a750b162ea41a359, user_id=19395692e62a407fa84b1449d299e1f4, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.787489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.807688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1136/2238] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:44 2022] PUT /identity/v3/projects/e9cc4a806ada4382a750b162ea41a359/users/19395692e62a407fa84b1449d299e1f4/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:44.825532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.826029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.826549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.859008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] MFA Rules not processed for user `19395692e62a407fa84b1449d299e1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:44.860884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.891339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:44.980370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1103/2239] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:44.986903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.107383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f42b8-c81e-4ba3-8bbf-04d50a6c7d0c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.115377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.116053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.116335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.116639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.117507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.122726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.153139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1137/2240] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:45.159225 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.185717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6381c-1d03-48ad-a0c5-8e6d4d568c2e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.293105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.294106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.294290 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.294649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.295237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.297281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.329276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1104/2241] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:45 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:41:45.334192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.361820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657281d-8eed-48c1-8dd4-ae11da3a19a8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.369695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.370091 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.370457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.370849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.371897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.374119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.389594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1138/2242] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:45.395476 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.410447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5a17-f673-4c69-9cf1-239269a6dad7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.415936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.416843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.417126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.417407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] PATH_INFO: `/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628/users/9ec9d356e1984257801983dc983de821/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.442100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8e74bb60cf4167b00cd1f2a0cf1628, user_id=9ec9d356e1984257801983dc983de821, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.445360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.482183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1105/2243] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:45 2022] PUT /identity/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628/users/9ec9d356e1984257801983dc983de821/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:45.487613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.499444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909147e4-886a-4a9b-b504-efc3d1925f2f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.526757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:45.582663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.583505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.583846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.584248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.585216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.587423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2b71a-612f-4734-8592-891c3ee5dac1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.608063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1139/2244] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:45.613139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.626239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09caaf-c250-4a21-aefb-53a4aeed095e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.631736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.632700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.632986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.633259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] PATH_INFO: `/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628/users/9ec9d356e1984257801983dc983de821/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.639382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8e74bb60cf4167b00cd1f2a0cf1628, user_id=9ec9d356e1984257801983dc983de821, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.642849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:45.658628 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1106/2245] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:45 2022] PUT /identity/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628/users/9ec9d356e1984257801983dc983de821/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:45.670265 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.670581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.670846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.702508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] MFA Rules not processed for user `9ec9d356e1984257801983dc983de821`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:45.704328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:45.823052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1140/2246] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:45.841760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:45.846717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:45.968761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e463241-7259-4af5-84e3-9f41fd729d1c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:45.972822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:45.973493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:45.973780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:45.974059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:45.985679 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:45.988597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef77b-9ec1-444d-8fd4-453f5c3b677d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:46.005403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1107/2247] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:46.303669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:46.306185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:46.325480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d771c-5547-46b6-b1fc-ff0182ff850f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:46.330203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:46.331273 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:46.331730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:46.332802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:46.437723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:46.440949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:46.454952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1141/2248] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:47.564404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:47.565203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:47.565203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:47.592687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] MFA Rules not processed for user `8bd1061f505d4249ae14c7cbf68a950a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:47.595976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:47.621021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:47.707909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1108/2249] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:56.446707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=80 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:57.700233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=16 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:08.831786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:08.835544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:08.969624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af119-8656-4543-b8bb-02fb1495651e service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:08.983727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:08.984900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:08.985414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:08.985912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:08.989814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.047374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:09.051349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c59e2-d608-40a4-b341-be27a0f94bf7 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:09.080883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1142/2250] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:09.539867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:09.619513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d576-2320-4f88-8011-a1374b71ecb7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:09.624193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:09.625672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:09.625952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:09.626226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] PATH_INFO: `/v3/users/19395692e62a407fa84b1449d299e1f4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:09.627482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19395692e62a407fa84b1449d299e1f4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:09.629452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:09.684016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19395692e62a407fa84b1449d299e1f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:09.692265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19395692e62a407fa84b1449d299e1f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:09.705732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19395692e62a407fa84b1449d299e1f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:09.706571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19395692e62a407fa84b1449d299e1f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:09.707193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19395692e62a407fa84b1449d299e1f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:09.721450 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1109/2251] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:09 2022] DELETE /identity/v3/users/19395692e62a407fa84b1449d299e1f4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:09.726172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e6240-105e-4d05-9916-0b26bc7e0d54 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:09.726591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e6240-105e-4d05-9916-0b26bc7e0d54 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:09.726963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e6240-105e-4d05-9916-0b26bc7e0d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:09.747332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-920e6240-105e-4d05-9916-0b26bc7e0d54 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:09.748821 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-920e6240-105e-4d05-9916-0b26bc7e0d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.828383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1143/2252] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:09.839635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:09.842442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.905104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:09.906555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:09.906555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:09.929962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:09.931254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.931600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57196411-0cbe-48d5-a8ec-a52586afadb5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:09.936579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:09.937982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:09.938678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:09.939326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:09.956652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:09.959427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:09.982176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1110/2253] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:10.003384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1144/2254] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:10.076891 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:10.079297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:10.139928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:10.329385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e3f92-13ea-480b-9080-31a4409e026d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:10.337724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:10.338451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:10.338755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:10.339128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:10.359609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:10.361441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bfcf6-edb4-42d9-b587-77d210b60b79 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:10.370281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:10.372536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:10.389848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1111/2255] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:42:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:42:10.398878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2958d-ff01-4edc-ac64-2c404441f017 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:10.410818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:10.411815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:10.412155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:10.412466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] PATH_INFO: `/v3/users/809e4d438c9747698bcf0f16af286879` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:10.417690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=809e4d438c9747698bcf0f16af286879)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:10.419713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:10.499370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:10.546021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809e4d438c9747698bcf0f16af286879'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:10.559699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809e4d438c9747698bcf0f16af286879'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:10.586287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809e4d438c9747698bcf0f16af286879'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:10.606987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809e4d438c9747698bcf0f16af286879'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:10.608310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809e4d438c9747698bcf0f16af286879'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:10.637189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:10.638093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:10.638609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:10.640977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1145/2256] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:10 2022] DELETE /identity/v3/users/809e4d438c9747698bcf0f16af286879 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:42:10.666641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:10.667949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:10.816941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1112/2257] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:10.832496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:10.835132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:10.846341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:10.848034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:10.874346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16230-c06b-4713-8823-a6ae5a576ec0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:10.898838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:10.900318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:10.900798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:10.901228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] PATH_INFO: `/v3/projects/e9cc4a806ada4382a750b162ea41a359` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:10.903381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cc4a806ada4382a750b162ea41a359)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:10.908400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:10.980205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e1c8-7ef1-4903-82ed-0a262df7dd07 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:10.993647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:10.994346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:10.994674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:10.994954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:11.016660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:11.018153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:11.037019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cc4a806ada4382a750b162ea41a359', 'request_id': 'req-29556af3-e63d-400f-aaa3-d0b6263726e1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.047519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1146/2258] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:11.051273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cc4a806ada4382a750b162ea41a359', 'request_id': 'req-29556af3-e63d-400f-aaa3-d0b6263726e1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.064401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cc4a806ada4382a750b162ea41a359', 'request_id': 'req-29556af3-e63d-400f-aaa3-d0b6263726e1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.087880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] The token cache is being invalidate because project e9cc4a806ada4382a750b162ea41a359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:11.088230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.090941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.093949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1113/2259] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:10 2022] DELETE /identity/v3/projects/e9cc4a806ada4382a750b162ea41a359 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:11.098826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:11.115169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85342b18-18df-429f-a692-8926fc9684f8 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:11.351739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:11.352904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:11.353448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:11.353935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] PATH_INFO: `/v3/users/9ec9d356e1984257801983dc983de821` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:11.366136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec9d356e1984257801983dc983de821)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:11.368796 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:11.505017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec9d356e1984257801983dc983de821'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.522204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:11.531723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec9d356e1984257801983dc983de821'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.561797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec9d356e1984257801983dc983de821'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.576590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec9d356e1984257801983dc983de821'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.578697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ec5d05b9-65df-48a6-bb2d-757e7a514f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec9d356e1984257801983dc983de821'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:11.596993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1147/2260] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:11 2022] DELETE /identity/v3/users/9ec9d356e1984257801983dc983de821 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:11.822328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:11.825564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:11.879767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556af3-e63d-400f-aaa3-d0b6263726e1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:12.039135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:12.119619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:12.121857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.129029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.129731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] PATH_INFO: `/v3/projects/aa3853573e1747c18d79bb90e297281b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.134282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3853573e1747c18d79bb90e297281b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:12.141636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:12.421915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3853573e1747c18d79bb90e297281b', 'request_id': 'req-337751c9-82a3-459e-bb66-21da5c33d6bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:12.443688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3853573e1747c18d79bb90e297281b', 'request_id': 'req-337751c9-82a3-459e-bb66-21da5c33d6bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:12.467377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd036a3-01d2-4412-975c-49dd558c1c05 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.468795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd036a3-01d2-4412-975c-49dd558c1c05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.468795 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd036a3-01d2-4412-975c-49dd558c1c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.480332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3853573e1747c18d79bb90e297281b', 'request_id': 'req-337751c9-82a3-459e-bb66-21da5c33d6bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:12.522368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] The token cache is being invalidate because project aa3853573e1747c18d79bb90e297281b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:12.523367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:12.525687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-337751c9-82a3-459e-bb66-21da5c33d6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:12.538251 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1114/2261] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:11 2022] DELETE /identity/v3/projects/aa3853573e1747c18d79bb90e297281b => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:42:12.544132 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.544960 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.545625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.583110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8fd036a3-01d2-4412-975c-49dd558c1c05 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:12.589994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8fd036a3-01d2-4412-975c-49dd558c1c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:12.618287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:12.625908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:12.846615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1115/2262] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:12.849406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:12.852075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:12.879105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838233b-5149-4572-a2d3-87d9eb666ce2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:12.893993 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1148/2263] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:12.897558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.898172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.898888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.904541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:12.912798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.913295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.913755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] PATH_INFO: `/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.921559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8e74bb60cf4167b00cd1f2a0cf1628)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:12.928523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:12.928917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:12.930120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:12.946562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1149/2264] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:12.948833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:12.955780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:13.082765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8e74bb60cf4167b00cd1f2a0cf1628', 'request_id': 'req-4dc13040-8a63-44c2-a3a7-9f6e665c786a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:13.098829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8e74bb60cf4167b00cd1f2a0cf1628', 'request_id': 'req-4dc13040-8a63-44c2-a3a7-9f6e665c786a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:13.108527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8e74bb60cf4167b00cd1f2a0cf1628', 'request_id': 'req-4dc13040-8a63-44c2-a3a7-9f6e665c786a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:13.111008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:13.143889 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] The token cache is being invalidate because project 6b8e74bb60cf4167b00cd1f2a0cf1628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:13.144221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:13.148068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:13.156181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1116/2265] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:12 2022] DELETE /identity/v3/projects/6b8e74bb60cf4167b00cd1f2a0cf1628 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:13.157110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.163862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:13.223537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770881f-1fbf-44fe-b2fb-7247d4ecdb60 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.312729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.313747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.314232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.315295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.328108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:13.343888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:13.429291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.432055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279a518-60d6-4335-9989-b1b6d07bd258 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.445134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1150/2266] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:42:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:42:13.447217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061308a4-0102-479e-90de-f1f35ba68924 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.447730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061308a4-0102-479e-90de-f1f35ba68924 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.448274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061308a4-0102-479e-90de-f1f35ba68924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.449957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13040-8a63-44c2-a3a7-9f6e665c786a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.464119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.465171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.465648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.466716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.468949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:13.469481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.471452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.480983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-061308a4-0102-479e-90de-f1f35ba68924 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:13.482749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-061308a4-0102-479e-90de-f1f35ba68924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:13.490559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1117/2267] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:42:13.495478 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.507953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c51ab-44e2-493b-95eb-e26a2ed39790 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.509794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1151/2268] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:13.514103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.514439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.514579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.514862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.515233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.515520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.515798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.516641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.519304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.559682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:13.565231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:13.566745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1118/2269] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:13 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:13.569248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.741507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34303b-983a-4022-958c-3480518258fc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.757546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.758246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.758564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.758845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.759740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.764120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.823547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1152/2270] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:13.826902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.834987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1119/2271] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:13 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:13.837969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.858728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f5220-77d8-4004-8ac7-23b282f1241f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.888055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.888055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.888055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.888055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.888807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:13.888807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.894692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df88a0-17a1-48f3-8aa9-dabd7b4181d8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.900081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.917989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.919846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:13.920196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:13.920505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:13.921326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:13.929059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:13.939716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1153/2272] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:42:13.947943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.957247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1120/2273] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:13.961014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:13.982302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc1e89-208d-4beb-863d-afba37d5a0a4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.985855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa299-da7b-463e-be3c-79701c618b72 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:13.998558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:13.999239 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.000142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.000549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.000742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.000881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.001012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.001161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.001862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.002046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.004613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.009049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.049168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1121/2274] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:14.050217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.058936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1154/2275] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:13 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:14.064468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.069360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a321f6-af00-4c36-adc4-98321a189bce admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:14.181704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836c313-8d53-4456-9931-e20996260dc8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:14.189221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.190160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.190923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.190923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.197083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.198533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:14.201003 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.316572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:14.338725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.339611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1155/2276] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:14 2022] POST /identity/v3/users => generated 284 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:14.339769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.339951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.340173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] PATH_INFO: `/v3/projects/edffd84b958d4e2a880fae7c4709d36d/users/6c7e0694c5ab46a4adef7a1f7b1fed4c/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.345328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.373161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda23190-bbc1-4a0f-98d9-71a60f621625 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:14.393111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.394261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.403935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edffd84b958d4e2a880fae7c4709d36d, user_id=6c7e0694c5ab46a4adef7a1f7b1fed4c, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.403935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018fa5-6677-4b04-b461-2ccc909d7447 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.407043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.407725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.409061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.412117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.428559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1156/2277] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:14.434680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.440780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1122/2278] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:14 2022] PUT /identity/v3/projects/edffd84b958d4e2a880fae7c4709d36d/users/6c7e0694c5ab46a4adef7a1f7b1fed4c/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:14.443518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.443919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.444305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.463070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b77a-87be-4ae0-986b-38a17800a2ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:14.496302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:14.506911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:14.660402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.661806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.662269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.662864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.664559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.666956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.683759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1157/2279] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:14.688293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.702800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68730d-6618-408c-a762-31a09f1b719f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:14.708440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:14.709893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.710349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.710840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] PATH_INFO: `/v3/projects/61ce43dd79bd41f3b26b68fbc75ea00e/users/2390d3682b1b41a48e0d21e3a5728124/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.729423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ce43dd79bd41f3b26b68fbc75ea00e, user_id=2390d3682b1b41a48e0d21e3a5728124, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:14.731897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1123/2280] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:14.735036 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0422b2-6ddc-4423-868f-38097ae73838 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:14.760206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:14.763095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:14.765766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1158/2281] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:14 2022] PUT /identity/v3/projects/61ce43dd79bd41f3b26b68fbc75ea00e/users/2390d3682b1b41a48e0d21e3a5728124/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:14.777708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.777981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.778268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.855082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] MFA Rules not processed for user `2390d3682b1b41a48e0d21e3a5728124`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:14.856404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:15.101142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa237-6748-4972-bda7-a837ea2153ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.116553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1159/2282] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:15.120400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:15.289355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:15.299871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.300970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.301381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.302325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.328531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:15.330827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:15.335386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ac39a-ddd0-40dc-a322-4701646da977 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.342970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.344032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.344705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.345472 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.346886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:15.350460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:15.380381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1124/2283] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:15.434376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1160/2284] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:15 2022] POST /identity/v3/projects => generated 367 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:15.439407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:15.471526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e37cf-fbfb-4a6a-8472-06c178558de0 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.498619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:15.645666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.646933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.647650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.648147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.649099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:15.651558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:15.685079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1125/2285] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:15 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:15.691785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:15.738317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929749-630c-474e-af4d-0620b7c0b026 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.819160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.826149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.837163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.837691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.839158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:15.842328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:15.921127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:15.929263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1161/2286] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:15.941235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b67471-ece2-4396-be53-a66ea7701911 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.953279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.956841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.958944 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.959433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] PATH_INFO: `/v3/projects/a347f9f1a70048fe858a33f1dd7b492a/users/1a3867e7761247c7b1a576dc1ed27da0/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.015148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a347f9f1a70048fe858a33f1dd7b492a, user_id=1a3867e7761247c7b1a576dc1ed27da0, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:16.025197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:16.079520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1126/2287] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:15 2022] PUT /identity/v3/projects/a347f9f1a70048fe858a33f1dd7b492a/users/1a3867e7761247c7b1a576dc1ed27da0/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:16.082457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:16.117192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f6dd8-587b-4a2f-9c63-6208fc979db1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:16.266371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:16.267364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:16.267816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:16.268261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.269213 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:16.270932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640841d0-3686-414e-86c6-41fd7461f9e0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:16.293039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1162/2288] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:16.299651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:16.321480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a597-ad63-4a4a-a984-11a83bc07f4f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:16.339677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:16.340602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:16.340886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:16.343588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] PATH_INFO: `/v3/projects/a347f9f1a70048fe858a33f1dd7b492a/users/1a3867e7761247c7b1a576dc1ed27da0/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.352033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a347f9f1a70048fe858a33f1dd7b492a, user_id=1a3867e7761247c7b1a576dc1ed27da0, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:16.363838 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:16.410443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1127/2289] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:16 2022] PUT /identity/v3/projects/a347f9f1a70048fe858a33f1dd7b492a/users/1a3867e7761247c7b1a576dc1ed27da0/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:16.427550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:16.427846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:16.428106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.493435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] MFA Rules not processed for user `1a3867e7761247c7b1a576dc1ed27da0`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:16.494768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:16.717815 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1163/2290] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:16.727176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:16.729305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:17.015758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241c0-9714-4bfc-b0a9-598efade9707 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:17.019776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:17.021263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:17.021573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:17.021860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:17.036921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:17.038945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:17.056176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1128/2291] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:17.324606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:17.342791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac91f7-28fd-40a9-9fbf-8fe007704adf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:17.350007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:17.350941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:17.351346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:17.351866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:17.479035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:17.508520 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:17.511409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9ab92-685d-489c-98db-03ada8b713b7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:17.539462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1164/2292] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:21.129736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:21.133775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:21.523911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c6c7-d7ba-47dc-b73a-84c5cbc954fb service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:21.535168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:21.536005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:21.536357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:21.536636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] PATH_INFO: `/v3/users/05eeb282e5de45568d06ab6b135fe343` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:21.542893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05eeb282e5de45568d06ab6b135fe343)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:21.546615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:21.583128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:21.770059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05eeb282e5de45568d06ab6b135fe343'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:21.793840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05eeb282e5de45568d06ab6b135fe343'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:21.839294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05eeb282e5de45568d06ab6b135fe343'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:21.841212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05eeb282e5de45568d06ab6b135fe343'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:21.842335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05eeb282e5de45568d06ab6b135fe343'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:21.899618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1129/2293] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:21 2022] DELETE /identity/v3/users/05eeb282e5de45568d06ab6b135fe343 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:21.906488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:21.907626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:21.908172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:21.983062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:21.988638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:22.134040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1165/2294] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:22.148243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:22.149852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:22.332567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc59b3c-f307-48a1-9599-9466505c958a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:22.348098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:22.348954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:22.349386 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:22.349687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:22.373930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:22.384724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0b98c-c557-49f1-8e16-25a00342a542 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:22.424303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1130/2295] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:24.114976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:24.119701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:24.182699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208d234-0feb-43dc-ac40-4808d53454cc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:24.262347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:24.264781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:24.265950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:24.266837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] PATH_INFO: `/v3/projects/4eb47c73399f4267962d7bdbbe95d19e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:24.269754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb47c73399f4267962d7bdbbe95d19e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:24.274891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:24.538543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb47c73399f4267962d7bdbbe95d19e', 'request_id': 'req-d2b7b2fd-0490-49ec-891a-bf56755fe335'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:24.571101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb47c73399f4267962d7bdbbe95d19e', 'request_id': 'req-d2b7b2fd-0490-49ec-891a-bf56755fe335'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:24.616086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb47c73399f4267962d7bdbbe95d19e', 'request_id': 'req-d2b7b2fd-0490-49ec-891a-bf56755fe335'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:24.714135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] The token cache is being invalidate because project 4eb47c73399f4267962d7bdbbe95d19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:24.718542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:24.723321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2b7b2fd-0490-49ec-891a-bf56755fe335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:24.736887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1166/2296] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:24 2022] DELETE /identity/v3/projects/4eb47c73399f4267962d7bdbbe95d19e => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:24.744923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:24.746319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:24.747031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:24.819565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:24.821906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:25.091399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1131/2297] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:25.105463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:25.106164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:25.107717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:25.179863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:25.192558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:25.250110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1167/2298] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:25.255118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:25.292458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cee8f-3d4b-4d02-85bc-1bc5f3f9123a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:25.309994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:25.312529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:25.316163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:25.316463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:25.324223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:25.324790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:25.335540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:25.377149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1132/2299] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:42:25.382248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:25.408878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6765f-1cd2-4016-b62c-9ed57d719f31 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:25.436002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:25.438491 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:25.441122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:25.442174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:25.448894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:25.457408 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:25.531679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1168/2300] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:25 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:25.537491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:25.550015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f751a-6ca5-4c57-bae4-f18bba5a4487 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:25.603505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:25.787305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:25.788879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:25.789424 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:25.789930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:25.791044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:25.793392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:25.849906 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1133/2301] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:25 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:25.857092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:25.926271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95153-5712-4288-b3c1-d5009134a90b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:25.943777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:25.945937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:25.946675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:25.947875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:25.951888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:25.959357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:26.002336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1169/2302] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:26.008021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:26.027513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86d157-2725-4590-b9c3-9307d2a91fa4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:26.035622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:26.037409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.037911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.038649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.039704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:26.042058 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fbeac-ef88-4db6-846d-34ac961d0a59 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:26.101837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:26.112870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1134/2303] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:26.156549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011582f-2fd0-4ce4-bf5a-0c56dc1c5f41 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:26.196142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:26.205762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.211159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.211986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] PATH_INFO: `/v3/projects/0716540bfe424998910050af002e65f0/users/5a58dfbf37dc48fea022bcd4d8cd712d/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.354995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0716540bfe424998910050af002e65f0, user_id=5a58dfbf37dc48fea022bcd4d8cd712d, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:26.371903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:26.522797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1170/2304] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:26 2022] PUT /identity/v3/projects/0716540bfe424998910050af002e65f0/users/5a58dfbf37dc48fea022bcd4d8cd712d/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:26.545199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.545199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.545199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.585904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:26.598288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:26.599555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] MFA Rules not processed for user `5a58dfbf37dc48fea022bcd4d8cd712d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:26.602309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:26.694718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:26.710169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1135/2305] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:26.746151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:26.749641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:26.904870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e3e17-a848-4965-b562-225fa41e4dd8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:26.909305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:26.910040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.910402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.910724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.941398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:26.950462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7db24d-48f8-4595-9bee-5b339328c60c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:26.968318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1136/2306] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:27.044855 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7df77-13af-491a-acb9-28fc1cb58cb9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:27.067648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:27.069009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:27.069550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:27.070063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] PATH_INFO: `/v3/users/0f069d8b472e4cb4807ada23855800c8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:27.073383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f069d8b472e4cb4807ada23855800c8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:27.095144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:27.245703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f069d8b472e4cb4807ada23855800c8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:27.266369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f069d8b472e4cb4807ada23855800c8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:27.330675 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f069d8b472e4cb4807ada23855800c8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:27.356829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f069d8b472e4cb4807ada23855800c8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:27.359638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f069d8b472e4cb4807ada23855800c8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:27.398879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1171/2307] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:26 2022] DELETE /identity/v3/users/0f069d8b472e4cb4807ada23855800c8 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:27.405448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:27.405824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:27.406021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:27.450292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:27.451584 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:27.499163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:27.559700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1137/2308] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:27.579250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:27.580838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:27.932580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca72fb-73a1-4151-bad3-4727e4675b0a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:27.954303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:27.962626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:27.975429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:27.976715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:28.058548 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:28.062300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988f02-5b74-450e-bb51-03709ebabaed service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:28.120070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1172/2309] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:28.406768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:28.408784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:28.464861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172982f-45ba-4f63-9d91-10a304b2a990 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:28.478921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:28.483545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:28.484420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:28.484940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] PATH_INFO: `/v3/users/2390d3682b1b41a48e0d21e3a5728124` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:28.487735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2390d3682b1b41a48e0d21e3a5728124)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:28.495143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:28.711357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2390d3682b1b41a48e0d21e3a5728124'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:28.721874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2390d3682b1b41a48e0d21e3a5728124'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:28.763170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2390d3682b1b41a48e0d21e3a5728124'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:28.764144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2390d3682b1b41a48e0d21e3a5728124'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:28.765122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2390d3682b1b41a48e0d21e3a5728124'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:28.802165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1138/2310] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:28 2022] DELETE /identity/v3/users/2390d3682b1b41a48e0d21e3a5728124 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:28.807139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:28.808478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:28.809256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:28.869695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:28.878937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:28.965291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:29.147174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1173/2311] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:29.568477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:29.570092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:29.770335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6503a6-4fe8-40e9-a5e9-c5a5b90f5feb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:29.782245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:29.782977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:29.783252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:29.783526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:29.843773 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:29.845691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:29.896870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1139/2312] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:30.514056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:30.516904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:30.555182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b21f7-0780-47b5-914b-75c3d88ec1fa None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:30.626426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:30.627350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:30.627635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:30.627915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] PATH_INFO: `/v3/projects/596431401ebd431492d52e760d961ede` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:30.630849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=596431401ebd431492d52e760d961ede)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:30.632870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:30.832638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596431401ebd431492d52e760d961ede', 'request_id': 'req-c0236199-0415-4985-9067-e67a80c942c9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:30.847925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596431401ebd431492d52e760d961ede', 'request_id': 'req-c0236199-0415-4985-9067-e67a80c942c9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:30.871083 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596431401ebd431492d52e760d961ede', 'request_id': 'req-c0236199-0415-4985-9067-e67a80c942c9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:30.896941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] The token cache is being invalidate because project 596431401ebd431492d52e760d961ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:30.897723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:30.898868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:30.901570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1174/2313] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:30 2022] DELETE /identity/v3/projects/596431401ebd431492d52e760d961ede => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:32.957551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:32.959337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.004974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f40d-26c3-4c5c-afec-88d9626412c4 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:33.198943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:33.252939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:33.257032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.257368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.257655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] PATH_INFO: `/v3/projects/61ce43dd79bd41f3b26b68fbc75ea00e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.260600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ce43dd79bd41f3b26b68fbc75ea00e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:33.262901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:33.412725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ce43dd79bd41f3b26b68fbc75ea00e', 'request_id': 'req-90c8915c-7b30-4599-9818-66466112770b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.466485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ce43dd79bd41f3b26b68fbc75ea00e', 'request_id': 'req-90c8915c-7b30-4599-9818-66466112770b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.479120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ce43dd79bd41f3b26b68fbc75ea00e', 'request_id': 'req-90c8915c-7b30-4599-9818-66466112770b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.511766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] The token cache is being invalidate because project 61ce43dd79bd41f3b26b68fbc75ea00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:33.512184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.514988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.519723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1140/2314] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:32 2022] DELETE /identity/v3/projects/61ce43dd79bd41f3b26b68fbc75ea00e => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:33.524991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:33.527969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.549708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0236199-0415-4985-9067-e67a80c942c9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:33.725801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:33.727422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.727982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.728510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] PATH_INFO: `/v3/users/1a3867e7761247c7b1a576dc1ed27da0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.730811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3867e7761247c7b1a576dc1ed27da0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:33.733737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:33.908108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3867e7761247c7b1a576dc1ed27da0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.925685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:33.938059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3867e7761247c7b1a576dc1ed27da0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:33.977592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3867e7761247c7b1a576dc1ed27da0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:34.000655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3867e7761247c7b1a576dc1ed27da0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:34.001571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-584e09d1-551d-4bc4-9291-d27c78447e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3867e7761247c7b1a576dc1ed27da0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:34.014946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1175/2315] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:33 2022] DELETE /identity/v3/users/1a3867e7761247c7b1a576dc1ed27da0 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:37.094101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:37.095606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:37.133647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8915c-7b30-4599-9818-66466112770b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:37.410202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:37.411581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:37.412009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:37.422528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] PATH_INFO: `/v3/projects/a347f9f1a70048fe858a33f1dd7b492a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:37.436512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a347f9f1a70048fe858a33f1dd7b492a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:37.450720 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:37.817721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a347f9f1a70048fe858a33f1dd7b492a', 'request_id': 'req-fda79e4b-25a0-41c9-bcc1-b2a497309a48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:37.861346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a347f9f1a70048fe858a33f1dd7b492a', 'request_id': 'req-fda79e4b-25a0-41c9-bcc1-b2a497309a48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:37.874279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a347f9f1a70048fe858a33f1dd7b492a', 'request_id': 'req-fda79e4b-25a0-41c9-bcc1-b2a497309a48'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:37.930472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] The token cache is being invalidate because project a347f9f1a70048fe858a33f1dd7b492a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:37.931232 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:37.932707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fda79e4b-25a0-41c9-bcc1-b2a497309a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:37.936140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1141/2316] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:37 2022] DELETE /identity/v3/projects/a347f9f1a70048fe858a33f1dd7b492a => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:37.943773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427677-3a1a-4204-a43d-8018524e287f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:37.944717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427677-3a1a-4204-a43d-8018524e287f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:37.944717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427677-3a1a-4204-a43d-8018524e287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:37.973453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36427677-3a1a-4204-a43d-8018524e287f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:37.975113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36427677-3a1a-4204-a43d-8018524e287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:38.158703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.159246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.159246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.160134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1176/2317] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:38.166431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:38.227376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:38.231269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:38.261121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427677-3a1a-4204-a43d-8018524e287f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:38.279462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1142/2318] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:38.293591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427677-3a1a-4204-a43d-8018524e287f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:38.305997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:38.308433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.314314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.316215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.317785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:38.319983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:38.322509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:38.358454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1177/2319] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:42:38.363675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:38.378567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0cc3-a861-4004-9cf5-868f9b223560 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:38.391805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:38.393332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.395900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.396379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.397536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:38.404576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:38.547603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:38.548449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1143/2320] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:38 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:38.568769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e18cb4-8316-4124-9605-84355a667c91 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:38.807712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:38.809839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.810640 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.812042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.813564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:38.817852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:38.889483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1178/2321] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:38 2022] POST /identity/v3/users => generated 288 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:38.892573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:38.895414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:38.899364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:38.925633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115dfa0-61bf-4480-a32e-702632de05ac admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:38.938661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f14ba7-f794-45b4-b24e-af4cbe4371b5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:38.942533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:38.944945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.945691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.947663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] PATH_INFO: `/v3/users/5a58dfbf37dc48fea022bcd4d8cd712d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.957304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a58dfbf37dc48fea022bcd4d8cd712d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:38.962849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:38.964945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:38.972298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:38.973376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:38.974142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:38.975900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:38.990367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:39.040129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1179/2322] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:39.042157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:39.083620 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c9ff-5925-4d33-aac7-7562df3f9638 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:39.107311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a58dfbf37dc48fea022bcd4d8cd712d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:39.114585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:39.119916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.120733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.121413 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.124701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:39.131658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a58dfbf37dc48fea022bcd4d8cd712d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:39.135141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:39.183236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1180/2323] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:39.183471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a58dfbf37dc48fea022bcd4d8cd712d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:39.184837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a58dfbf37dc48fea022bcd4d8cd712d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:39.185302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:39.187869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83c2998c-c14c-4b49-a6b0-090b42d9e382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a58dfbf37dc48fea022bcd4d8cd712d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:39.190730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:39.236075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1144/2324] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:38 2022] DELETE /identity/v3/users/5a58dfbf37dc48fea022bcd4d8cd712d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:42:39.238941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f72e-ffcb-4fba-add4-b88b7cea5a7f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.239242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f72e-ffcb-4fba-add4-b88b7cea5a7f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.241653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f72e-ffcb-4fba-add4-b88b7cea5a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.252380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc771ba-9895-4038-a22b-7bc737a31053 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:39.294218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ec26f72e-ffcb-4fba-add4-b88b7cea5a7f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:39.300676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ec26f72e-ffcb-4fba-add4-b88b7cea5a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:39.343169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:39.506971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1145/2325] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:39.524910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374516-c86e-4dd6-9ddb-053e32c3cbfb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.525470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374516-c86e-4dd6-9ddb-053e32c3cbfb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.527426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374516-c86e-4dd6-9ddb-053e32c3cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.581443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:39.586216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.592540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.595270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] PATH_INFO: `/v3/projects/2ac56aacdaa04d01a6ec533d0b1a4e6a/users/7b0e05d3a0b8485db66b974fb4f33a99/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.595539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-23374516-c86e-4dd6-9ddb-053e32c3cbfb None None] MFA Rules not processed for user `6c7e0694c5ab46a4adef7a1f7b1fed4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:39.597783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-23374516-c86e-4dd6-9ddb-053e32c3cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:39.789921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac56aacdaa04d01a6ec533d0b1a4e6a, user_id=7b0e05d3a0b8485db66b974fb4f33a99, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:39.795681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:39.795926 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:39.858699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1181/2326] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:39 2022] PUT /identity/v3/projects/2ac56aacdaa04d01a6ec533d0b1a4e6a/users/7b0e05d3a0b8485db66b974fb4f33a99/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:39.858699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:39.872518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:39.922120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1146/2327] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:39.924282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.924616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.924937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.019794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] MFA Rules not processed for user `7b0e05d3a0b8485db66b974fb4f33a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:40.020975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:40.075817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6fc6-78c2-4db6-a1aa-b322a6efdf59 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.086485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.087188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.087476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.087759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.164056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1147/2328] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:40.164694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.167533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:40.252539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.254049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.304195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1182/2329] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:40.305562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.353394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f384-cf38-4361-82f8-b9f303b4ca7c service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.371627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.373829 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.374512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.376231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.381187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.385168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.391494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170100d8-6de9-48df-b201-ce1d334c1745 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.398569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.399600 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.399992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.400660 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.419171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.422566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.477104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1183/2330] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:40 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:40.481132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.525895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1148/2331] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:40 2022] POST /identity/v3/projects => generated 379 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:40.531260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.649271 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:40.762847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:40.792187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd228d3-8319-4ca8-b10a-8e55b9f3a96e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.818751 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.822466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f53b2-813d-4fd7-8fe1-168003e84365 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.823306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.823616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.823897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.824657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.836559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.847281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.852789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.853143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.853391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.857814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.863776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.909473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1184/2332] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:40 2022] POST /identity/v3/users => generated 289 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:40.913860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.929882 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1149/2333] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:40 2022] POST /identity/v3/users => generated 296 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:40.939497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:40.951901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2509a2-6e59-4de5-b0f3-99cc119c5e9f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.972921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d72f7d-1ed8-457b-a395-19610b1851c6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.976188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.977270 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.977775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.978464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.980127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.982036 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.988969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.989951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.990434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.997166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.999766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.002828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.029966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1185/2334] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:41.031327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.053632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1150/2335] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:41.055055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.095319 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc55e7-74ae-46a4-be6f-03edc69181d2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:41.124863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d046e-2ad4-4ce2-88cf-cb53a4b35eb0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:41.134261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:41.147808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.148409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.148905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] PATH_INFO: `/v3/projects/774a83196bf843c4aedd7519b0a44dca/users/d2a021c05c0744eaa6a282b1e37292fa/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.177622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:41.184177 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.184811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.185317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.191004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.193710 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.219362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774a83196bf843c4aedd7519b0a44dca, user_id=d2a021c05c0744eaa6a282b1e37292fa, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.225328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.279358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1151/2336] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:41 2022] PUT /identity/v3/projects/774a83196bf843c4aedd7519b0a44dca/users/d2a021c05c0744eaa6a282b1e37292fa/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:41.288114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.307191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1186/2337] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:41.308530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.369229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded69f2-5267-48af-9e4c-c2795fc8a75a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:41.469986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721fdf-0ba9-40a4-854c-58050f339b97 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:41.494546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:41.495252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.495538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.495813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.496578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.501279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.531371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1152/2338] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:41.532385 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.575383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490e043-fa2e-4ad0-9826-5e7309e81f09 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:41.619259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:41.627022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.629626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.631953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] PATH_INFO: `/v3/projects/774a83196bf843c4aedd7519b0a44dca/users/d2a021c05c0744eaa6a282b1e37292fa/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.640674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=774a83196bf843c4aedd7519b0a44dca, user_id=d2a021c05c0744eaa6a282b1e37292fa, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.653285 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.706989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1153/2339] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:41 2022] PUT /identity/v3/projects/774a83196bf843c4aedd7519b0a44dca/users/d2a021c05c0744eaa6a282b1e37292fa/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:41.710025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:41.731603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:41.734466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.735604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.736027 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:41.737011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] PATH_INFO: `/v3/projects/b95240b00dfa416e82c86930dce6e25c/users/24e42a6fe8b04486955d9205670894a3/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.824218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95240b00dfa416e82c86930dce6e25c, user_id=24e42a6fe8b04486955d9205670894a3, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:41.830186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbea1d-fee2-4a34-bc53-469d399441a6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:41.885382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1187/2340] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:41 2022] PUT /identity/v3/projects/b95240b00dfa416e82c86930dce6e25c/users/24e42a6fe8b04486955d9205670894a3/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:41.933848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:41.934697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:41.935209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:41.952865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1f7b-0c27-4b4a-b666-98e181e091fe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:42.039510 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] MFA Rules not processed for user `24e42a6fe8b04486955d9205670894a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:42.040779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:42.117653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:42.229269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:42.231235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.231844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.232472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] PATH_INFO: `/v3/projects/0716540bfe424998910050af002e65f0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.235019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0716540bfe424998910050af002e65f0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:42.246825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:42.321860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1188/2341] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:42.326303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:42.353883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77ee7c-468a-4bd1-88e6-a2fe162be369 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:42.370180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:42.371061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.371477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.371784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.372644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:42.375865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:42.448667 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:42.468350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0716540bfe424998910050af002e65f0', 'request_id': 'req-788cf902-a3f7-4b1e-b3ce-24782ada515a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:42.489294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0716540bfe424998910050af002e65f0', 'request_id': 'req-788cf902-a3f7-4b1e-b3ce-24782ada515a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:42.512890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0716540bfe424998910050af002e65f0', 'request_id': 'req-788cf902-a3f7-4b1e-b3ce-24782ada515a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:42.526687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1189/2342] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:42 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:42.528242 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:42.560260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] The token cache is being invalidate because project 0716540bfe424998910050af002e65f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:42.561893 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:42.563956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-788cf902-a3f7-4b1e-b3ce-24782ada515a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:42.570040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1154/2343] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:41 2022] DELETE /identity/v3/projects/0716540bfe424998910050af002e65f0 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:42.577938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265f86-c1ba-48f0-af5b-8b5a209577c2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.578338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265f86-c1ba-48f0-af5b-8b5a209577c2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.578766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265f86-c1ba-48f0-af5b-8b5a209577c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.669066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55265f86-c1ba-48f0-af5b-8b5a209577c2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:42.671776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55265f86-c1ba-48f0-af5b-8b5a209577c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:42.758900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce879-98d8-4158-84fd-7156ad8b03b8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:42.786846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:42.787834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.788318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.788756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.789701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:42.808441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:42.834967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1155/2344] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:42.847920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.848520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.849467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.890733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1190/2345] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:42 2022] POST /identity/v3/users => generated 295 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:42.895754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:42.915342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:42.948422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a7fe1-1152-44f1-aec6-7a7ce28d4345 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:42.967160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:42.970026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:42.971209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.971638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.972025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.972470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:42.973998 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:42.977204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:42.992394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1156/2346] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:42.997816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.001253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1191/2347] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:43.006969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.016915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd1488-873b-4486-8cd0-e179adcf51c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.028466 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c73fc-1ca9-4b0f-8ef2-ccd7eb93eb5b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.037184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.037947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.038241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.038916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.040249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:43.040249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.042387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.042666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.047248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.047759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.048305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] PATH_INFO: `/v3/projects/05f19ba7ebb24218af03d1823a6b9050/users/900416d7d89940188562ed300c31374c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.077380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1157/2348] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:42:43.077582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f19ba7ebb24218af03d1823a6b9050, user_id=900416d7d89940188562ed300c31374c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.077582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.078597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.098069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1615b2-e05e-49b7-a20c-20d74051bebb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.126247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1192/2349] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:43 2022] PUT /identity/v3/projects/05f19ba7ebb24218af03d1823a6b9050/users/900416d7d89940188562ed300c31374c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:43.133165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.306594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:43.358569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.363770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.363770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.363770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.363770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.379785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.477135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898ca91-72f3-49f4-b2f4-bd6f55c1ac7a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.485130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1158/2350] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:43 2022] POST /identity/v3/projects => generated 385 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:43.490794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.517747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.519216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.519966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.520591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.522110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.525305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.610523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1193/2351] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:43.615707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.699462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac276-bc5e-454e-85a3-abe5dff64254 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.710823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62b193-fe47-46dd-b351-876110b9098e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.715539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.725851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.726700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.727166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] PATH_INFO: `/v3/projects/05f19ba7ebb24218af03d1823a6b9050/users/900416d7d89940188562ed300c31374c/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.729722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f19ba7ebb24218af03d1823a6b9050, user_id=900416d7d89940188562ed300c31374c, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.735365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad352a-a27e-422d-9d75-b740525082ca admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.738955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.739666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.739955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.740236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.740951 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.748045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.755428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1194/2352] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:43 2022] PUT /identity/v3/projects/05f19ba7ebb24218af03d1823a6b9050/users/900416d7d89940188562ed300c31374c/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:43.799528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.800881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.801344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.835452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1159/2353] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:43 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:43.862714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:43.962808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] MFA Rules not processed for user `900416d7d89940188562ed300c31374c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:43.965817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:43.966032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3db885-2158-409b-a01d-152d99931bbf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:44.022803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:44.113355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:44.115314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:44.117819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:44.117819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:44.118443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:44.120551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:44.137972 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1160/2354] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:42:44.142014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:44.177856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357fbd-b83a-44b9-be91-d42a8a9bb1c5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:44.209271 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:44.211440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:44.214539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:44.217722 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:44.220786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:44.227337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:44.269757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1161/2355] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:42:44.281651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1195/2356] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:44.287614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:44.292110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:44.297004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:44.330452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da11e14-5ce8-4384-a34a-2c1626b2a276 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:44.362981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:44.365104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:44.365772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:44.366448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] PATH_INFO: `/v3/projects/dd5b5b8f1d9e4115924c5bc8cc0ef572/users/5c3816080cc245b5869e21bf772feb4b/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:44.385847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938c8f-8daf-417a-b95d-65f4913c4cbf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:44.391575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:44.392692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:44.393143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:44.393912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:44.397970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:44.446946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5b5b8f1d9e4115924c5bc8cc0ef572, user_id=5c3816080cc245b5869e21bf772feb4b, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:44.459388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dbde1-05e6-4509-b9b8-73bacefaa91f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:44.524594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1162/2357] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:42:44 2022] PUT /identity/v3/projects/dd5b5b8f1d9e4115924c5bc8cc0ef572/users/5c3816080cc245b5869e21bf772feb4b/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:44.607538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:44.610015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:44.611006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:44.628975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:44.634237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:44.681460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] MFA Rules not processed for user `5c3816080cc245b5869e21bf772feb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:44.701798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:44.712534 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1196/2358] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:44.765794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:44.955892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1163/2359] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:44.970353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:44.973030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:45.035169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa47e0-b36d-4f09-90fe-63452c9b56b4 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:45.329137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:45.330498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:45.334860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:45.335547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:45.404462 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:45.407529 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:45.471187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1197/2360] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:47.100530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:47.153641 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173a8a0-ccf4-4e52-8c3f-8dce29ba2c02 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:47.171700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:47.173062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:47.175147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:47.175767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:47.183989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:47.373843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:47.375927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:47.391820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1164/2361] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:49.184744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:49.203876 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5596-93b1-4c2c-9944-8735b2250416 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:49.221562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:49.222949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:49.223421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:49.223996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:49.245279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:49.331133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:49.520350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:49.522697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1ef28-2010-4d99-a397-97c85c8ee074 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:49.568682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1198/2362] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:49.575249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440de-f67f-4d5e-8662-55276f8701b7 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:49.596646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:49.598142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:49.600718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:49.600917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] PATH_INFO: `/v3/users/5c3816080cc245b5869e21bf772feb4b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:49.602452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3816080cc245b5869e21bf772feb4b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:49.607992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:49.780265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3816080cc245b5869e21bf772feb4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:49.788989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:49.793172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3816080cc245b5869e21bf772feb4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:49.838567 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3816080cc245b5869e21bf772feb4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:49.841943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3816080cc245b5869e21bf772feb4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:49.844458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3816080cc245b5869e21bf772feb4b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:49.878430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1165/2363] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:49 2022] DELETE /identity/v3/users/5c3816080cc245b5869e21bf772feb4b => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:42:49.880296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f897-5b3e-4f06-b371-9e85508fa7c3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:49.880944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f897-5b3e-4f06-b371-9e85508fa7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:49.881235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f897-5b3e-4f06-b371-9e85508fa7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:49.957179 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-69f5f897-5b3e-4f06-b371-9e85508fa7c3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:49.966994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69f5f897-5b3e-4f06-b371-9e85508fa7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:50.146945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1199/2364] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:50.158609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:50.160437 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:50.297022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e215a8-c402-45a0-b022-da1b597756dc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:50.325186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:50.329545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:50.330082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:50.331696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:50.381549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:50.392321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cec26d-3a29-4b6b-bf74-2616ee97dff9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:50.447654 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1166/2365] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:50.454071 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:50.460898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:50.461365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:50.470233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1200/2366] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:42:50 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:50.482849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53936-3758-4590-8130-7950826abf0a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:50.483494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53936-3758-4590-8130-7950826abf0a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:50.486022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53936-3758-4590-8130-7950826abf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:50.533638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b1a53936-3758-4590-8130-7950826abf0a None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:50.537991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b1a53936-3758-4590-8130-7950826abf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:50.783289 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1167/2367] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:42:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:50.792193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:50.795319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:51.045712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199604f-3872-4cb5-8700-effc6f1ecf23 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:51.066196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:51.067728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:51.068411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:51.069157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:51.120965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:51.125146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bbd12-5cac-4c50-8896-2ca651bf60b9 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:51.189011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1201/2368] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:51.432560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271671-85bd-436b-be1a-886a03179b27 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:51.433860 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271671-85bd-436b-be1a-886a03179b27 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:51.433992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271671-85bd-436b-be1a-886a03179b27 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:51.437826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1168/2369] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:42:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:51.444600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daf19-f0e7-4878-9090-1af3e9a497ad None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:51.444933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daf19-f0e7-4878-9090-1af3e9a497ad None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:51.445616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daf19-f0e7-4878-9090-1af3e9a497ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:51.552695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c03daf19-f0e7-4878-9090-1af3e9a497ad None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:51.563933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c03daf19-f0e7-4878-9090-1af3e9a497ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:51.611674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1202/2370] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:42:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:51.633241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271671-85bd-436b-be1a-886a03179b27 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:51.720330 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271671-85bd-436b-be1a-886a03179b27 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:51.730227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:51.735303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:51.736296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:51.738559 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:51.797831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:51.800228 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:51.833861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1169/2371] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:52.132992 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f355b0-eaec-45cf-9f4c-62782c1acd77 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:52.134180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:52.135227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f355b0-eaec-45cf-9f4c-62782c1acd77 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:52.136001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f355b0-eaec-45cf-9f4c-62782c1acd77 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:52.140195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:52.141823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1203/2372] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:42:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:52.159207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:52.160281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:52.164139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:52.222272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288e82-73ec-44da-9a78-5ba4050ba794 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:52.240326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:52.243067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:52.244048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:52.245057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:52.247850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:52.248329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] PATH_INFO: `/v3/projects/dd5b5b8f1d9e4115924c5bc8cc0ef572` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:52.252673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5b5b8f1d9e4115924c5bc8cc0ef572)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:52.256295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:52.269587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1204/2373] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:42:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:52.286282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:52.329001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3408c-b357-4e41-ade1-ea50de426c83 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:52.346969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:52.350909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:52.351410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:52.351923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:52.403436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:52.408551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9197c-43f3-4002-9f09-9994a948e0cf service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:52.435034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:52.444121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5b5b8f1d9e4115924c5bc8cc0ef572', 'request_id': 'req-5083ed3f-fcb3-4776-8721-373d3d6cddda'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:52.450654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1205/2374] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:52.488033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5b5b8f1d9e4115924c5bc8cc0ef572', 'request_id': 'req-5083ed3f-fcb3-4776-8721-373d3d6cddda'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:52.503496 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5b5b8f1d9e4115924c5bc8cc0ef572', 'request_id': 'req-5083ed3f-fcb3-4776-8721-373d3d6cddda'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:52.548404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] The token cache is being invalidate because project dd5b5b8f1d9e4115924c5bc8cc0ef572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:52.548824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:52.549858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:52.552148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1170/2375] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:52 2022] DELETE /identity/v3/projects/dd5b5b8f1d9e4115924c5bc8cc0ef572 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:53.632963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:53.633577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:53.633577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:53.686690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] MFA Rules not processed for user `d2a021c05c0744eaa6a282b1e37292fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:53.688527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:53.704861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:53.931149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1206/2376] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:53.948033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:53.955230 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:54.078081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ed3f-fcb3-4776-8721-373d3d6cddda admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:54.089092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:54.090099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:54.090582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:54.091030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:54.093530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:54.188996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:54.190851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113f1a0-e419-43c8-a53c-836436d9701a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:54.205920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1171/2377] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:54.817069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:54.818674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:54.926606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c68ee2-2dea-4b64-90d7-454a7db5cc55 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:54.934258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:54.935604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:54.936164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:54.936638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] PATH_INFO: `/v3/users/7b0e05d3a0b8485db66b974fb4f33a99` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:54.938699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0e05d3a0b8485db66b974fb4f33a99)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:54.941129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:55.031017 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0e05d3a0b8485db66b974fb4f33a99'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:55.086769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0e05d3a0b8485db66b974fb4f33a99'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:55.104597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0e05d3a0b8485db66b974fb4f33a99'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:55.117345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0e05d3a0b8485db66b974fb4f33a99'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:55.120225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0e05d3a0b8485db66b974fb4f33a99'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:55.135994 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1207/2378] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:54 2022] DELETE /identity/v3/users/7b0e05d3a0b8485db66b974fb4f33a99 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:55.151516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c485a3-6e96-42cb-b7ae-d0acff364873 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:55.153372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c485a3-6e96-42cb-b7ae-d0acff364873 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:55.154693 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c485a3-6e96-42cb-b7ae-d0acff364873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:55.193178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d2c485a3-6e96-42cb-b7ae-d0acff364873 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:55.196306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d2c485a3-6e96-42cb-b7ae-d0acff364873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:55.351037 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1172/2379] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:55.363336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:55.367853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:55.414381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:55.473602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088d0e-9d7d-4301-b450-20222c72a918 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:55.485183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:55.486327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:55.487013 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:55.487940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:55.511031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:55.514880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:55.542007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1208/2380] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:55.606359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:55.606687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:55.607016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:55.633460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:55.637064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:55.651883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:55.790634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1173/2381] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:55.819587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:55.821897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:56.007148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68f67-758c-4f58-8751-398ced8bbc54 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:56.025580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:56.026601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:56.027411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:56.027803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:56.056605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:56.058256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:56.101314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1209/2382] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:42:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:42:56.223948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:56.227301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:56.275508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456c75c-9b14-44b7-92bc-a5c4b57b9107 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:56.295276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:56.299038 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:56.299687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:56.300644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] PATH_INFO: `/v3/projects/2ac56aacdaa04d01a6ec533d0b1a4e6a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:56.304875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac56aacdaa04d01a6ec533d0b1a4e6a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:56.310573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:56.452791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac56aacdaa04d01a6ec533d0b1a4e6a', 'request_id': 'req-b4607984-9439-48dd-9b1c-92b59186582f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.468847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac56aacdaa04d01a6ec533d0b1a4e6a', 'request_id': 'req-b4607984-9439-48dd-9b1c-92b59186582f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.485078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac56aacdaa04d01a6ec533d0b1a4e6a', 'request_id': 'req-b4607984-9439-48dd-9b1c-92b59186582f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.505354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] The token cache is being invalidate because project 2ac56aacdaa04d01a6ec533d0b1a4e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:56.506885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.506885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.509904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1174/2383] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:56 2022] DELETE /identity/v3/projects/2ac56aacdaa04d01a6ec533d0b1a4e6a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:56.514555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:56.527229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749593a5-32cc-4b1c-bf7d-c21ce23d738a service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:56.619266 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:56.620382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:56.620684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:56.620964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] PATH_INFO: `/v3/users/24e42a6fe8b04486955d9205670894a3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:56.623166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e42a6fe8b04486955d9205670894a3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:56.626073 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:56.739770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e42a6fe8b04486955d9205670894a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.759976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e42a6fe8b04486955d9205670894a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.781114 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:56.805963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e42a6fe8b04486955d9205670894a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.824062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e42a6fe8b04486955d9205670894a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.825464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-31981d52-41bd-4313-a7d8-6af6cd916ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e42a6fe8b04486955d9205670894a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:56.844109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1210/2384] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:56 2022] DELETE /identity/v3/users/24e42a6fe8b04486955d9205670894a3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:57.389945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:57.392358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:57.426169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607984-9439-48dd-9b1c-92b59186582f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:57.524026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:57.524343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:57.524649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:57.528488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:57.529382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:57.529664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:57.529941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] PATH_INFO: `/v3/projects/b95240b00dfa416e82c86930dce6e25c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:57.531287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95240b00dfa416e82c86930dce6e25c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:57.533254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:57.555793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:57.557869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:57.590358 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:57.601280 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95240b00dfa416e82c86930dce6e25c', 'request_id': 'req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.613662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95240b00dfa416e82c86930dce6e25c', 'request_id': 'req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.625874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95240b00dfa416e82c86930dce6e25c', 'request_id': 'req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.645018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1211/2385] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:42:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:57.645880 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] The token cache is being invalidate because project b95240b00dfa416e82c86930dce6e25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:57.646362 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.647749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.651139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1175/2386] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:57 2022] DELETE /identity/v3/projects/b95240b00dfa416e82c86930dce6e25c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:57.657001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:57.660226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:57.672724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:57.676813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:57.692448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d82a-f6ac-4b5b-9b51-f99ccdc5a4ac None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:57.799208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:57.801087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:57.801261 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bad6a-2c8a-491e-a0e3-2a6575a0689c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:57.801485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:57.802187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] PATH_INFO: `/v3/users/900416d7d89940188562ed300c31374c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:57.804327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=900416d7d89940188562ed300c31374c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:57.806996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:57.810725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:57.811723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:57.812005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:57.812268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:57.814767 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:57.900183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900416d7d89940188562ed300c31374c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.910245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900416d7d89940188562ed300c31374c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.931594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900416d7d89940188562ed300c31374c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.943725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900416d7d89940188562ed300c31374c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.944404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900416d7d89940188562ed300c31374c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:57.953552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:57.958429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d469f-abcb-4732-ae61-04d0c8f21614 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:57.960352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1212/2387] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:57 2022] DELETE /identity/v3/users/900416d7d89940188562ed300c31374c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:57.993752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1176/2388] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:42:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:42:58.868890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:58.870692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:58.880421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:58.991085 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1e38-bc01-4b1e-92b2-25cdfb302e9f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:59.006202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:59.008861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.010042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.010840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] PATH_INFO: `/v3/projects/05f19ba7ebb24218af03d1823a6b9050` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.014168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f19ba7ebb24218af03d1823a6b9050)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:59.022700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:59.179043 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f19ba7ebb24218af03d1823a6b9050', 'request_id': 'req-7d046e69-2337-4bab-bb91-38f6de8ab8f9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:59.199661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f19ba7ebb24218af03d1823a6b9050', 'request_id': 'req-7d046e69-2337-4bab-bb91-38f6de8ab8f9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:59.217800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f19ba7ebb24218af03d1823a6b9050', 'request_id': 'req-7d046e69-2337-4bab-bb91-38f6de8ab8f9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:59.247569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] The token cache is being invalidate because project 05f19ba7ebb24218af03d1823a6b9050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:42:59.247988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:59.249726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7d046e69-2337-4bab-bb91-38f6de8ab8f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:42:59.253372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1213/2389] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:42:58 2022] DELETE /identity/v3/projects/05f19ba7ebb24218af03d1823a6b9050 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:59.260303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.260868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.260868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.314446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:59.314446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:59.452669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1177/2390] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:59.465343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.465637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.465922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.514891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:59.517109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:59.527310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1214/2391] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:59.532179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:59.544252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3193b-1304-446e-9185-248ef37d7c3f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:59.552553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:59.553381 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.553701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.554011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.561822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:42:59.562262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:59.564456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:59.595732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:59.605341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1178/2392] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:42:59.610094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:59.625404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672711a-79e3-424b-aef2-758e01e86c0f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:59.631260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:59.631942 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.632229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.632508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.633885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:59.637614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:59.682678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1215/2393] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:42:59 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:42:59.686922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:59.700165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d8480-2666-4bfa-ab91-9ffa356a3d18 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:59.877871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:59.881701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.882086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.882554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.883422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:59.891277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:59.958360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1179/2394] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:42:59 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:42:59.959627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:59.977514 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769cb3d-8606-4106-bb74-8d3c703b8250 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:59.989923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:59.995333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:59.996133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:59.997030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:59.997030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:59.997203 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:59.997931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:00.001882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:00.020677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf12f8-38d4-449c-960e-d80f7584b6ee admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:00.022221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1216/2395] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:43:00.027728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:00.052475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:00.052740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e4fae-976e-459a-bf60-413a9b6f828b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:00.053819 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.054212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.054642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] PATH_INFO: `/v3/users/33e56d53506d49a68c864556486c94c4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.058596 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e56d53506d49a68c864556486c94c4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:00.061226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:00.071648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:00.072861 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.073365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.073809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.074922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:00.081455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:00.113488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1217/2396] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:00.118050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:00.154858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e3e2-c38c-4220-85ae-9525d7e8c2f7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:00.167434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:00.170600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.171081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.171947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] PATH_INFO: `/v3/projects/e4a2b573669141b4a714b7d70d8f7fac/users/09825b4817824225a4a0b41169097d42/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.208173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e56d53506d49a68c864556486c94c4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:00.217237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e56d53506d49a68c864556486c94c4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:00.233760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e56d53506d49a68c864556486c94c4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:00.235065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e56d53506d49a68c864556486c94c4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:00.236234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ca3599d1-e9fd-43fc-a660-7328442858d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e56d53506d49a68c864556486c94c4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:00.244628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a2b573669141b4a714b7d70d8f7fac, user_id=09825b4817824225a4a0b41169097d42, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:00.249039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:00.277114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1180/2397] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:42:59 2022] DELETE /identity/v3/users/33e56d53506d49a68c864556486c94c4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:00.293041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1218/2398] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:00 2022] PUT /identity/v3/projects/e4a2b573669141b4a714b7d70d8f7fac/users/09825b4817824225a4a0b41169097d42/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:00.304504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.304822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.305133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.352183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] MFA Rules not processed for user `09825b4817824225a4a0b41169097d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:00.354263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:00.503129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:00.544934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1181/2399] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:00.548065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:00.552208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:00.637759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:00.806899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733d84-9ecf-4b5f-9537-a8f3bd79ba42 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:00.930299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:00.931022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.931377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.931641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.932604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:00.940556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:00.997998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.000832 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:01.027422 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1219/2400] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:01.028976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.038664 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a01df3-8710-4bed-88d9-e365525c8377 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.175031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e888be-9b39-4057-8d55-a15d6aeef0e8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.178149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.179286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.180305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.180771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] PATH_INFO: `/v3/projects/a907861f2aca4a4286e9a46cdae6c4d2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.184139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a907861f2aca4a4286e9a46cdae6c4d2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.186919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.191358 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.199801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.200240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.201664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.202521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.208045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.254813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1220/2401] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:43:01 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:01.261716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.297128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6b14-3e6d-4976-b127-85e4742d3437 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.318441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a907861f2aca4a4286e9a46cdae6c4d2', 'request_id': 'req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:01.349153 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a907861f2aca4a4286e9a46cdae6c4d2', 'request_id': 'req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:01.364302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a907861f2aca4a4286e9a46cdae6c4d2', 'request_id': 'req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:01.395615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] The token cache is being invalidate because project a907861f2aca4a4286e9a46cdae6c4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:43:01.396067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:01.396942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce3447fa-5ea5-4efa-a85c-eb6240b034a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:01.412446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1182/2402] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:43:00 2022] DELETE /identity/v3/projects/a907861f2aca4a4286e9a46cdae6c4d2 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:43:01.421414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5a5cf-a193-4fb7-8627-deb106d387dc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.421879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5a5cf-a193-4fb7-8627-deb106d387dc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.422359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5a5cf-a193-4fb7-8627-deb106d387dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.456856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:01.492604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ca5a5cf-a193-4fb7-8627-deb106d387dc None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:01.493887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ca5a5cf-a193-4fb7-8627-deb106d387dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:01.543471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.545093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.545093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.545483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.546341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.570614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.614539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1221/2403] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:01.630893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.645903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:01.661580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1183/2404] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:01.666539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.667115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.667404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.709520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:01.710909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:01.719010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30a3d4-c4e4-493d-b6c8-546aa6fc7923 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.731326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.733182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.733182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.733182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.733893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.739060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1184/2405] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:01.742672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.743653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.769708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1222/2406] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:01.775035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688c629-e33f-4cc7-844c-c32d90228d75 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.775741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.810481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed9078-2211-4646-9891-85f6515c9a13 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.815111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.815754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.816115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.816297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.822611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:01.826562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:01.826921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:01.827205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] PATH_INFO: `/v3/projects/f2752b49d61246589390a1aea1efa7e8/users/2b806b4ebe604063a179e0c360d792f4/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:01.832364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:43:01.832691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.834383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.872383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:01.898192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1185/2407] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:43:01.899129 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:01.899849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2752b49d61246589390a1aea1efa7e8, user_id=2b806b4ebe604063a179e0c360d792f4, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:01.903315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabab8e7-06e1-41c0-abf8-09e97ef1fb3e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:01.933839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113c6ce-a5a9-4c2e-ab79-60df2776825e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:01.934831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1223/2408] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:01 2022] PUT /identity/v3/projects/f2752b49d61246589390a1aea1efa7e8/users/2b806b4ebe604063a179e0c360d792f4/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:02.030767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d7944-499b-46fc-8626-02162918c240 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.031484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d7944-499b-46fc-8626-02162918c240 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.032115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d7944-499b-46fc-8626-02162918c240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.096636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d99d7944-499b-46fc-8626-02162918c240 None None] MFA Rules not processed for user `2b806b4ebe604063a179e0c360d792f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:02.098974 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d99d7944-499b-46fc-8626-02162918c240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:02.103104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.103806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.104096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.104471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.105085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.112190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.184466 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1186/2409] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:43:01 2022] POST /identity/v3/projects => generated 361 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:02.186235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.299997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1224/2410] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:02.301414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d7944-499b-46fc-8626-02162918c240 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.326463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d7944-499b-46fc-8626-02162918c240 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.374445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d988c-e4f3-440f-a3cb-a51ebd778968 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.388848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.389867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.390302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.390834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.391752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.394070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.460356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1187/2411] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:43:02 2022] POST /identity/v3/users => generated 281 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:02.461730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.498150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088aa8-e9ac-4e17-ae08-4142b835966f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.510877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.512432 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.513682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.514228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.515423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.518042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.533854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.535134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.535752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.536353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.537571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.541998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.567016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1188/2412] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:02.571295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.594382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1225/2413] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:43:02 2022] POST /identity/v3/projects => generated 383 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:02.596058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.623095 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a368094-f518-43dd-874a-d4d64e6b3967 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.663103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:02.784915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a69de-e433-4271-8d7a-6b09758821c9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.790647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.791330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.791621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.791903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.793022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.795645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.829605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.831398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.831656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.831933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.832895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.835188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.847108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1226/2414] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:43:02 2022] POST /identity/v3/users => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:02.847883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:02.855093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.867058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26121-250c-44f9-ac62-73d83ecc4869 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.878924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1189/2415] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:02.879476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.880639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.880639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.880908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.880999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.881473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:02.883168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:02.898111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1227/2416] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:02.899509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:02.904035 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c55-0f09-4e87-b2b6-10d5421f0043 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.923156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.924504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.924991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.925482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] PATH_INFO: `/v3/projects/67d57a4d5d50445baf86ce244f874b06/users/9e706f89259641c48915392dc40eceab/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:02.953717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad156-e2d5-4825-858c-b31b2ce9bbea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:02.972214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:02.973641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:02.974223 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:02.974812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] PATH_INFO: `/v3/projects/2d884512b1ee4047a9b04fe94c436c67/users/4a48efbdbdf5403da2cd40bb7d4504a9/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.003755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d57a4d5d50445baf86ce244f874b06, user_id=9e706f89259641c48915392dc40eceab, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:03.013929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c5890-fe11-41c4-b2bc-3d4836238619 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:03.015409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d884512b1ee4047a9b04fe94c436c67, user_id=4a48efbdbdf5403da2cd40bb7d4504a9, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:03.018464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:03.042708 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1228/2417] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:02 2022] PUT /identity/v3/projects/2d884512b1ee4047a9b04fe94c436c67/users/4a48efbdbdf5403da2cd40bb7d4504a9/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:03.047254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:03.053323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1190/2418] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:02 2022] PUT /identity/v3/projects/67d57a4d5d50445baf86ce244f874b06/users/9e706f89259641c48915392dc40eceab/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:43:03.064866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.066538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.067243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.137775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] MFA Rules not processed for user `9e706f89259641c48915392dc40eceab`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:03.141872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.163637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73e8b-40e1-4704-a34c-4bbe1a32c47a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:03.277821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1191/2419] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:03.294803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:03.298350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.389554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead5dc7-d5f6-40b7-8748-4577d2ff182d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:03.397012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:03.398789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.399785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.400701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.414555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:03.416395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.416956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.417834 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.419904 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:03.421556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee5cef-65de-4ee5-9c52-a2a98183a976 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:03.421887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:03.424229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:03.440665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1192/2420] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:03.446530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1229/2421] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:43:03.453016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:03.466384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07aad3-7c71-442f-9b7d-2cbebb0d193d service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:03.473204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:03.475515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.476102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.476758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] PATH_INFO: `/v3/projects/2d884512b1ee4047a9b04fe94c436c67/users/4a48efbdbdf5403da2cd40bb7d4504a9/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.479564 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d884512b1ee4047a9b04fe94c436c67, user_id=4a48efbdbdf5403da2cd40bb7d4504a9, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:03.502021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:03.532000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.532373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.532702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.534537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1193/2422] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:03 2022] PUT /identity/v3/projects/2d884512b1ee4047a9b04fe94c436c67/users/4a48efbdbdf5403da2cd40bb7d4504a9/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:03.569732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] MFA Rules not processed for user `4a48efbdbdf5403da2cd40bb7d4504a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:03.571603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.577198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:03.827585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1230/2423] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:03.840922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:03.843257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.976072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462039d8-df7b-49a7-aad7-ee80d02c9cae admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:03.980849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:03.982078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.982366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.982673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.984678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.997598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:04.075912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:04.079122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:04.114822 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1194/2424] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:06.335110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:06.344693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a66e2-91ee-4cca-8b27-9c0d139defc4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:06.348959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:06.349768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:06.350119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:06.350495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:06.452425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:06.455155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:06.473446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1231/2425] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:07.380056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:07.423268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14d5f-7640-4c28-b7a3-89225dfe611f service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:07.436487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:07.446967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:07.447201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:07.447794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:07.473548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:07.475060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148a98f-db88-4ab9-bfed-c59d61723aab service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:07.500667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1195/2426] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:43:11.997178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:12.082443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c2e90-03f8-4dd1-aa2b-5c986b5f07f1 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:12.093828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:12.096176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.097275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.098000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] PATH_INFO: `/v3/users/09825b4817824225a4a0b41169097d42` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.101364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09825b4817824225a4a0b41169097d42)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:12.105097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:12.165400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:12.176773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09825b4817824225a4a0b41169097d42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.186208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09825b4817824225a4a0b41169097d42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.213127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09825b4817824225a4a0b41169097d42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.225063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09825b4817824225a4a0b41169097d42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.225801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09825b4817824225a4a0b41169097d42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.234132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1232/2427] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:43:11 2022] DELETE /identity/v3/users/09825b4817824225a4a0b41169097d42 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:12.239041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.239653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.239862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.261995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:12.261995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:12.323296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1196/2428] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:12.336451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:12.338536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:12.404119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570426d-f23d-4299-b548-a1c3757b670f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:12.408112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:12.408778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.409056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.409477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.420696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:12.422149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:12.436810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1233/2429] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:12.745541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:12.749265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:12.765808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c67972-6be7-4ae7-9af1-ba78e88a39a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:12.773995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:12.774941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.775234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.777407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] PATH_INFO: `/v3/projects/e4a2b573669141b4a714b7d70d8f7fac` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.779547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a2b573669141b4a714b7d70d8f7fac)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:12.781543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:12.838634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a2b573669141b4a714b7d70d8f7fac', 'request_id': 'req-33d72052-084d-4827-a6ca-924ebd034041'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.849015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a2b573669141b4a714b7d70d8f7fac', 'request_id': 'req-33d72052-084d-4827-a6ca-924ebd034041'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.856507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a2b573669141b4a714b7d70d8f7fac', 'request_id': 'req-33d72052-084d-4827-a6ca-924ebd034041'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.867886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] The token cache is being invalidate because project e4a2b573669141b4a714b7d70d8f7fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:43:12.868178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.868851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:12.870399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1197/2430] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:43:12 2022] DELETE /identity/v3/projects/e4a2b573669141b4a714b7d70d8f7fac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:12.875586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:12.891346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1f22b-6e54-43fd-bb2f-424f2aa1fe17 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:12.962919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:12.963809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.964096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.964652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] PATH_INFO: `/v3/users/2b806b4ebe604063a179e0c360d792f4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.966027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b806b4ebe604063a179e0c360d792f4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:12.968456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:13.035045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b806b4ebe604063a179e0c360d792f4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.045744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b806b4ebe604063a179e0c360d792f4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.056582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:13.070577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b806b4ebe604063a179e0c360d792f4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.079959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b806b4ebe604063a179e0c360d792f4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.080614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b806b4ebe604063a179e0c360d792f4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.087282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1234/2431] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:43:12 2022] DELETE /identity/v3/users/2b806b4ebe604063a179e0c360d792f4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:43:13.519772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:13.521907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:13.539099 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d72052-084d-4827-a6ca-924ebd034041 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:13.560762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:13.609119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:13.610029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:13.610307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:13.610678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] PATH_INFO: `/v3/projects/f2752b49d61246589390a1aea1efa7e8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:13.612524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2752b49d61246589390a1aea1efa7e8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:13.614507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:13.665418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2752b49d61246589390a1aea1efa7e8', 'request_id': 'req-65636635-6535-468a-bcfe-6d3c9d5ecde0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.675775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2752b49d61246589390a1aea1efa7e8', 'request_id': 'req-65636635-6535-468a-bcfe-6d3c9d5ecde0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.683253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2752b49d61246589390a1aea1efa7e8', 'request_id': 'req-65636635-6535-468a-bcfe-6d3c9d5ecde0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.695557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] The token cache is being invalidate because project f2752b49d61246589390a1aea1efa7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:43:13.695847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.696521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.698103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1198/2432] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:43:13 2022] DELETE /identity/v3/projects/f2752b49d61246589390a1aea1efa7e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:13.702394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:13.703928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:13.717729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3803a4-e4c1-4b57-8ae4-2c33bd9e8cd7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:13.785651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:13.786706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:13.787031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:13.787474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] PATH_INFO: `/v3/users/4a48efbdbdf5403da2cd40bb7d4504a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:13.788827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a48efbdbdf5403da2cd40bb7d4504a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:13.791002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:13.842969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a48efbdbdf5403da2cd40bb7d4504a9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.850377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a48efbdbdf5403da2cd40bb7d4504a9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.864966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a48efbdbdf5403da2cd40bb7d4504a9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.879995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a48efbdbdf5403da2cd40bb7d4504a9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.881274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5d2e0b79-c9b8-400e-87a2-9ad8fc151ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a48efbdbdf5403da2cd40bb7d4504a9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:13.888680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1235/2433] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:43:13 2022] DELETE /identity/v3/users/4a48efbdbdf5403da2cd40bb7d4504a9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:14.193787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.196204 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:14.214186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636635-6535-468a-bcfe-6d3c9d5ecde0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.285448 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.286501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.286874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.287183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] PATH_INFO: `/v3/projects/2d884512b1ee4047a9b04fe94c436c67` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.288561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d884512b1ee4047a9b04fe94c436c67)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.292092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.358828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d884512b1ee4047a9b04fe94c436c67', 'request_id': 'req-74416cd0-38e0-450e-ade6-58d4e1e981cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:14.363340 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:14.368653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d884512b1ee4047a9b04fe94c436c67', 'request_id': 'req-74416cd0-38e0-450e-ade6-58d4e1e981cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:14.381869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d884512b1ee4047a9b04fe94c436c67', 'request_id': 'req-74416cd0-38e0-450e-ade6-58d4e1e981cf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:14.395756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] The token cache is being invalidate because project 2d884512b1ee4047a9b04fe94c436c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:43:14.396125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:14.396779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74416cd0-38e0-450e-ade6-58d4e1e981cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:14.398517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1199/2434] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:43:14 2022] DELETE /identity/v3/projects/2d884512b1ee4047a9b04fe94c436c67 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:14.403948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.404788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.404788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.424610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:14.426141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:14.495809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:14.510243 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1236/2435] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:14.525281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.526136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.526816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.551550 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:14.552931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:14.562761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1200/2436] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:14.568063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.585186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a2952-2d92-4c8d-944d-3d35bc03c058 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.591916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.592967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.593471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.593941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.594939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:43:14.595515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.598251 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.617283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1237/2437] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:43:14.623454 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.635118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635440a-84cf-4357-a0ca-d5062aeef977 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.643548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.644716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.645215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.645753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.647397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.651263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.702436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1201/2438] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:43:14 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:43:14.708616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.724859 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29a188-2bab-463f-a653-0f70236c3707 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.805314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.806010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.806297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.806605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.807325 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.809851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.833410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1238/2439] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:43:14 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:43:14.837969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.853069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0af22-6640-4960-86f7-341947a1c291 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.860655 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.861799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.862294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.862818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.864175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.867236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.887116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1202/2440] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:14.897338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.908051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8b25d-821d-4f60-95d4-a677553e4935 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.912982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.913672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.913962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.914244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.915040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.916682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c306f3-cf6f-4e2c-aee8-9409bbc6d66b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.933478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1239/2441] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:43:14.938915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:14.950253 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ef6a3-38c0-4e6d-85f5-a55d6342d949 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:14.955629 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:14.956703 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.957033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.957374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] PATH_INFO: `/v3/projects/56792f3625e2457bbe9aa0dfa9763028/users/5c7220e14fcd40c9a78bbcef1a1db5d9/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.974573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56792f3625e2457bbe9aa0dfa9763028, user_id=5c7220e14fcd40c9a78bbcef1a1db5d9, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:14.977575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:14.994849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1203/2442] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:14 2022] PUT /identity/v3/projects/56792f3625e2457bbe9aa0dfa9763028/users/5c7220e14fcd40c9a78bbcef1a1db5d9/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:43:15.004185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e048-e352-4128-8a6a-c3bcc6c2035c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:15.004691 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e048-e352-4128-8a6a-c3bcc6c2035c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:15.005187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e048-e352-4128-8a6a-c3bcc6c2035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:15.037238 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1de0e048-e352-4128-8a6a-c3bcc6c2035c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:15.039566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1de0e048-e352-4128-8a6a-c3bcc6c2035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:15.114779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1240/2443] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:15.129315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:15.131782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:15.208554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31450-0a5c-4beb-adb8-54b0dab0b484 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:15.212547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:15.213215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:15.213498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:15.213778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:15.230243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:15.233861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:15.249468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1204/2444] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:43:16.284053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10ea31-7bc5-40fa-b953-b890206125f8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:16.284714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10ea31-7bc5-40fa-b953-b890206125f8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:16.284714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10ea31-7bc5-40fa-b953-b890206125f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:16.317430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3a10ea31-7bc5-40fa-b953-b890206125f8 None None] MFA Rules not processed for user `5c7220e14fcd40c9a78bbcef1a1db5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:16.318976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a10ea31-7bc5-40fa-b953-b890206125f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:16.393479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:16.404841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1241/2445] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:16.417830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:16.421391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:16.512576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0623cf-bf2b-49d1-846a-6bc3b75b9d32 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:16.516489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:16.517143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:16.517425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:16.517956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:16.529990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:16.532281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dd411-9dc0-4f24-93e8-dd15cdfd9092 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:16.547987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1205/2446] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:18.002830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aad14-930e-4f9c-8341-054210a30ee9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:18.003528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aad14-930e-4f9c-8341-054210a30ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:18.003528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aad14-930e-4f9c-8341-054210a30ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:18.022363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bf1aad14-930e-4f9c-8341-054210a30ee9 None None] MFA Rules not processed for user `5c7220e14fcd40c9a78bbcef1a1db5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:18.023845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bf1aad14-930e-4f9c-8341-054210a30ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:18.031917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1242/2447] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:22.716695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:22.718163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:22.718925 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:22.756045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] MFA Rules not processed for user `6c7e0694c5ab46a4adef7a1f7b1fed4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:22.759499 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:22.800356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:22.909007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1206/2448] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:24.653233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b1f-9816-43aa-af90-c5494bff08c8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:24.654230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b1f-9816-43aa-af90-c5494bff08c8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:24.654230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b1f-9816-43aa-af90-c5494bff08c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:24.685458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b63d9b1f-9816-43aa-af90-c5494bff08c8 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:24.687343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b63d9b1f-9816-43aa-af90-c5494bff08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:24.798361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1243/2449] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:24.803942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:24.805828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:24.952886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008533c7-b15d-4031-9c83-50ad31078145 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:24.966867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:24.967747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:24.970574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:24.971106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:25.001732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:25.009500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8353b7-9e18-44de-ac65-fca3ce340405 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:25.026104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1207/2450] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:43:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:43:34.763866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=30 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:35.019890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=47 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:41.791506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:41.793503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:41.793985 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:41.824641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] MFA Rules not processed for user `8bd1061f505d4249ae14c7cbf68a950a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:41.826234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:41.975096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1244/2451] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:45.319825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91605-df09-4b6a-9b57-f977f4e09e1d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:45.320697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91605-df09-4b6a-9b57-f977f4e09e1d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:45.320992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91605-df09-4b6a-9b57-f977f4e09e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:45.360680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b9a91605-df09-4b6a-9b57-f977f4e09e1d None None] MFA Rules not processed for user `ab3be59ea4c74da3b4e19273102bccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:45.362005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b9a91605-df09-4b6a-9b57-f977f4e09e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:45.493535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1208/2452] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:46.923890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:46.926132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:46.944184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32777b74-8aa8-4530-a105-5907fc618f9a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:46.954835 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:46.956253 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:46.956688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:46.956818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] PATH_INFO: `/v3/users/9e706f89259641c48915392dc40eceab` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:46.963419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e706f89259641c48915392dc40eceab)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:46.965595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:47.048355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e706f89259641c48915392dc40eceab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:47.060006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e706f89259641c48915392dc40eceab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:47.077225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e706f89259641c48915392dc40eceab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:47.090639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e706f89259641c48915392dc40eceab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:47.091468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e706f89259641c48915392dc40eceab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:47.099598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1245/2453] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:43:46 2022] DELETE /identity/v3/users/9e706f89259641c48915392dc40eceab => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:47.105380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:47.105953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:47.105953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:47.126104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:47.127455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:47.219128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1209/2454] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:47.234900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:47.237283 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:47.340440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec8100-317a-46c8-8156-157c89e04891 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:47.346676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:47.347753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:47.348217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:47.348673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:47.372524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:47.374149 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6958f-cbdd-4117-b632-f0feac50c390 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:47.387424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1246/2455] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:43:48.145109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:48.148652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:48.182242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c8294-b3af-43f7-80b4-cc5beee67377 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:48.194368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:48.203837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.205472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.206218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] PATH_INFO: `/v3/projects/67d57a4d5d50445baf86ce244f874b06` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.209145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d57a4d5d50445baf86ce244f874b06)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:48.213867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:48.313022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d57a4d5d50445baf86ce244f874b06', 'request_id': 'req-685f9b55-52dc-4208-906f-0b14b4fd87f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:48.328439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d57a4d5d50445baf86ce244f874b06', 'request_id': 'req-685f9b55-52dc-4208-906f-0b14b4fd87f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:48.336958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d57a4d5d50445baf86ce244f874b06', 'request_id': 'req-685f9b55-52dc-4208-906f-0b14b4fd87f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:48.354873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] The token cache is being invalidate because project 67d57a4d5d50445baf86ce244f874b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:43:48.355399 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:48.356367 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-685f9b55-52dc-4208-906f-0b14b4fd87f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:43:48.358177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1210/2456] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:43:48 2022] DELETE /identity/v3/projects/67d57a4d5d50445baf86ce244f874b06 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:43:48.365986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.366766 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.367045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.407425 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:48.410449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:48.441168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:48.565610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1247/2457] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:48.574052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.574553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.574900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.600989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:48.603140 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:48.622189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1211/2458] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:48.624338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:48.638922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0dd6f-1052-4759-91e9-3efa228f0393 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:48.657893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:48.658661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.659022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.659218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.660001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:43:48.660319 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:48.662062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:48.677062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1248/2459] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:43:48.681723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:48.693214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20a04-15de-4c2a-8e78-665b496a5c29 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:48.701074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:48.702059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.702912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.705233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.706681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:48.709881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:48.769867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:48.787977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:48.793071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1212/2460] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:43:48 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:48.812066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb457cd0-6919-4bd8-a04e-fc9b9774ca63 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:48.963939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:48.965637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:48.965927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:48.966559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:48.968094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:48.978655 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.036187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1249/2461] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:43:48 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:43:49.041162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.072714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fcf8-080d-4c63-879d-e364ae653495 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.081432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.082208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.082969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.083432 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.085143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.087765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.121492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1213/2462] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:43:49.125906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.157287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b98c62-7a56-4b88-9637-ee0dde691146 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.167588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.169038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.169747 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.170553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.172506 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.175984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e9315-ac4c-41cb-a35e-b1d0fe26b841 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.199007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1250/2463] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:43:49.203059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.224588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d66803-fcea-48a6-aefb-11afd1891666 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.235415 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.238096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.239201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.239952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] PATH_INFO: `/v3/projects/fdb60c3122d54267b88422f2da4afa8a/users/c4edf85ecfe44730b38fafe714190fa6/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.280680 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb60c3122d54267b88422f2da4afa8a, user_id=c4edf85ecfe44730b38fafe714190fa6, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.288310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.315219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1214/2464] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:43:49 2022] PUT /identity/v3/projects/fdb60c3122d54267b88422f2da4afa8a/users/c4edf85ecfe44730b38fafe714190fa6/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:43:49.327565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.328281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.329100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.381486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] MFA Rules not processed for user `c4edf85ecfe44730b38fafe714190fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:49.383485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:49.480565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1251/2465] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:49.498072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.502006 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:49.601024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec18585-cb63-4eae-8e6a-8720988ae771 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.606042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.607048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.607423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.607708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.619792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.622056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.630799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.633276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:49.633423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1215/2466] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:49.660171 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:49.741726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83935673-95a4-4d45-a574-b7a0f034c923 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.748284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.749445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.749963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.750526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.754049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:49.875923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.878968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af50f-f8ff-4b3b-80c2-01d0e76616ad service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.902883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1252/2467] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:43:59.628862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:59.890996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=43 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:01.587471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:01.668935 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419ea58-fcc3-4bf9-96fe-3dc14a921542 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:01.674798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:01.675985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:01.676382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:01.676667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] PATH_INFO: `/v3/users/c4edf85ecfe44730b38fafe714190fa6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:01.677977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4edf85ecfe44730b38fafe714190fa6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:01.680114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:01.737498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4edf85ecfe44730b38fafe714190fa6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:01.746046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4edf85ecfe44730b38fafe714190fa6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:01.761275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4edf85ecfe44730b38fafe714190fa6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:01.762198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4edf85ecfe44730b38fafe714190fa6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:01.763161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4edf85ecfe44730b38fafe714190fa6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:01.783053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1216/2468] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:44:01 2022] DELETE /identity/v3/users/c4edf85ecfe44730b38fafe714190fa6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:44:01.788248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:01.789011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:01.789403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:01.809966 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:01.811372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:01.881502 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1253/2469] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:01.892305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:01.894135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:01.966182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff5ac-5b3f-486d-aef7-df9120cdf611 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:01.970247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:01.970985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:01.971301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:01.971560 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:01.985343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:01.986978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:01.999325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1217/2470] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:44:02.745637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:02.748051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:02.781308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02012a6f-8f34-4913-a5d0-9798cb07a8b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:02.793686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:02.795883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:02.796417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:02.797120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] PATH_INFO: `/v3/projects/fdb60c3122d54267b88422f2da4afa8a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:02.799403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb60c3122d54267b88422f2da4afa8a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:02.801999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:02.904081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb60c3122d54267b88422f2da4afa8a', 'request_id': 'req-33996a8c-fab0-473e-b2a0-7d3cff44b65a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:02.913871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb60c3122d54267b88422f2da4afa8a', 'request_id': 'req-33996a8c-fab0-473e-b2a0-7d3cff44b65a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:02.927486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb60c3122d54267b88422f2da4afa8a', 'request_id': 'req-33996a8c-fab0-473e-b2a0-7d3cff44b65a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:02.949132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] The token cache is being invalidate because project fdb60c3122d54267b88422f2da4afa8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:44:02.949558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:02.950707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:44:02.953716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1254/2471] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:44:02 2022] DELETE /identity/v3/projects/fdb60c3122d54267b88422f2da4afa8a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:44:11.993978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:12.942905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:26.606125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:26.607788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:26.674078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e865-c4b6-4214-8561-62dab0c14bab service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:26.733986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:26.734672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:26.734957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:26.735376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:26.737327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:26.797688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:26.801854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:26.833986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:26.835546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:26.849531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1218/2472] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:26.897641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33996a8c-fab0-473e-b2a0-7d3cff44b65a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:26.903787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:26.904453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:26.904758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:26.905028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:26.906982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:26.973802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:26.975500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd1c5d-d172-4ecf-84fd-88caddd3f482 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:26.986293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1255/2473] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:27.176619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:27.178188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:27.191139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659e2c9-2af0-44de-9c61-1cc00c08058f service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:27.196776 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:27.197445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:27.198365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:27.198809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:27.200894 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:27.219118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:27.221974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:27.244831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1219/2474] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:36.981348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=56 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:37.234931 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=80 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:39.428084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:39.428695 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:39.429109 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:39.461065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] MFA Rules not processed for user `4909d4aef302404290adcd3268e53c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:39.463352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:39.476776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1256/2475] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:44:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:39.831434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:39.833016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:39.896491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870d90-de46-4fdb-8d05-22350dd45c14 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:39.900766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:39.901426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:39.901707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:39.901988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:39.903866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:39.914484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:39.916649 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920907a-ec45-4bed-848b-38cf72e4b948 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:39.927517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1220/2476] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:40.617489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:40.620988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:40.731983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798d95c-938d-4aa8-8472-a1a6cc9d599f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:40.740687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:40.741840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:40.742354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:40.743400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:40.746218 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:40.767938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:40.770789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:40.789052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1257/2477] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:45.840183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:45.841710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:45.842260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:45.869640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] MFA Rules not processed for user `5c7220e14fcd40c9a78bbcef1a1db5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:45.871332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:45.997377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1221/2478] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:46.022982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:46.040965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7164d-a68e-44db-b565-06c91d0b1dc8 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:46.045822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:46.046540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:46.047010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:46.047981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:46.050307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:46.061957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:46.063631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:46.076162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1258/2479] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:46.347344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:46.372464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab87f-7854-479d-b16c-7fa5a37f1ca7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:46.381245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:46.382155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:46.382626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:46.383751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:46.386046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:46.522898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:46.525431 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:46.540268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1222/2480] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:55.760742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:55.763380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:55.785881 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058ca16-bc39-49ba-9221-cfef4b896b3f service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:55.795767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:55.797162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:55.798011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:55.798807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:55.802722 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:55.819424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:55.824092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:55.848539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1259/2481] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:56.454170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:56.476392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456832-7227-4bbf-8291-83464cccac6c service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:56.489270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:56.490656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:56.491268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:56.491853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:56.495791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:56.522570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:56.526258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:56.548697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1223/2482] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:44:59.277231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:59.346400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecaa6-8210-44d6-9815-f07aea94b90d service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:59.371074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:59.375025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:59.376105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:59.376964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:59.382383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:59.596301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:59.600680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:59.627482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1260/2483] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:03.259842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:03.359913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f351-35aa-423a-baa4-504e8f765a03 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:03.374312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:03.375579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:03.376069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:03.377612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:03.382396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:03.391921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:03.450459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:03.453284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:03.473677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1224/2484] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:09.613879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=75 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:13.464744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:14.978581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:15.040212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a374-5a4e-4c77-86a6-c4d3aa8ab27e service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:15.044263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:15.044950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:15.045246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:15.045525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:15.047800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:15.164561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:15.166191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:15.191166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1261/2485] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:45:19.047185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:19.086188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4986c3-e85e-4378-bbf4-dd6810a01b5a service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:19.094799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:19.096342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:19.097054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:19.097530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:19.101608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:19.149639 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:19.158640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:19.213685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1225/2486] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:19.284700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:19.286684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:19.315862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad178c7-1474-4d01-b458-9c6f70d75df3 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:19.325411 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:19.326533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:19.327201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:19.327556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] PATH_INFO: `/v3/users/ab3be59ea4c74da3b4e19273102bccd9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:19.329519 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3be59ea4c74da3b4e19273102bccd9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:19.333033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:19.426284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3be59ea4c74da3b4e19273102bccd9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.450000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3be59ea4c74da3b4e19273102bccd9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.475457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3be59ea4c74da3b4e19273102bccd9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.488775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3be59ea4c74da3b4e19273102bccd9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.489962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a4127be2-0029-48d7-87d6-e7eaf71fd3f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3be59ea4c74da3b4e19273102bccd9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.498433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1262/2487] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:45:19 2022] DELETE /identity/v3/users/ab3be59ea4c74da3b4e19273102bccd9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:45:19.735517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:19.753801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5699-f3a6-4402-a9ca-c031ff0a8792 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:19.832647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:19.833704 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:19.834116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:19.834593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] PATH_INFO: `/v3/projects/afb65421f9504a918b7a47451fac6707` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:19.836742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb65421f9504a918b7a47451fac6707)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:19.839495 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:19.893384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb65421f9504a918b7a47451fac6707', 'request_id': 'req-d2c954e9-5a6e-4e2a-9259-826aac0336a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.910119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb65421f9504a918b7a47451fac6707', 'request_id': 'req-d2c954e9-5a6e-4e2a-9259-826aac0336a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.918611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb65421f9504a918b7a47451fac6707', 'request_id': 'req-d2c954e9-5a6e-4e2a-9259-826aac0336a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.931924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] The token cache is being invalidate because project afb65421f9504a918b7a47451fac6707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:45:19.932258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.932884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d2c954e9-5a6e-4e2a-9259-826aac0336a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:19.934651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1226/2488] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:45:19 2022] DELETE /identity/v3/projects/afb65421f9504a918b7a47451fac6707 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:45:19.940385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:19.940768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:19.941153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:19.965499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:20.044515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:20.045777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:20.113924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1263/2489] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:20.123051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.123754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.123998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.154218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:20.156056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:20.168318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1227/2490] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:45:20.174037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.191887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c8911-acc2-4e7c-9880-1c21c49bb1c6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.200915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.202131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.202750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.203336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.204767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:20.205393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.208560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.228849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1264/2491] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:20.234240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.244481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5a39-d9ff-45c6-a315-8cdbed61ae7a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.254480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.255173 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.255459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.255738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.256459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.259102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.310240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1228/2492] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:20 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:45:20.314929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.326882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5124-15b6-4882-9f6d-c6bb23c2236b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.397053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.397666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.397990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.398277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.399033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.401033 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.424217 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1265/2493] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:20 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:20.429254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.442316 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b380-2f7f-458b-9792-d248f0b4d39d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.449270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.449965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.450248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.450566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.451345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.453054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.464806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1229/2494] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:20.469341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.480169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111265f-c0b1-4fea-b370-c0c34bda499e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.485153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.485844 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.486129 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.486503 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.487252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.488894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d4ef-a9a4-467f-87b3-a8cd8c9200a4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.502275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1266/2495] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:20.506815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.519749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7974d-edfd-464a-893b-cab3ffccf2b3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.525236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.526212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.526538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.526817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] PATH_INFO: `/v3/projects/628c00a1a5df481982b083fa3c447be2/users/d66dde93e0f14ce799c9c6294f216781/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.543372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628c00a1a5df481982b083fa3c447be2, user_id=d66dde93e0f14ce799c9c6294f216781, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.546079 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.557182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:20.577336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1230/2496] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:20 2022] PUT /identity/v3/projects/628c00a1a5df481982b083fa3c447be2/users/d66dde93e0f14ce799c9c6294f216781/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:20.583362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8495b-fd5b-4860-bbd6-3251bedf27d3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.583656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8495b-fd5b-4860-bbd6-3251bedf27d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.583929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8495b-fd5b-4860-bbd6-3251bedf27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.606964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bab8495b-fd5b-4860-bbd6-3251bedf27d3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:20.608228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bab8495b-fd5b-4860-bbd6-3251bedf27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:20.662108 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:20.686255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1267/2497] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:20.697522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:20.699602 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:20.785430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4ef8-6ca3-4a00-8a55-fe450f9b1ff3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:20.789868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:20.790667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:20.791028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:20.791397 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:20.802954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:20.804623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:20.817096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1231/2498] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:45:27.126780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277605-9525-4381-a818-e59bd8b417d4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:27.127335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277605-9525-4381-a818-e59bd8b417d4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:27.127892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277605-9525-4381-a818-e59bd8b417d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:27.162889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-69277605-9525-4381-a818-e59bd8b417d4 None None] MFA Rules not processed for user `d66dde93e0f14ce799c9c6294f216781`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:27.164459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69277605-9525-4381-a818-e59bd8b417d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:27.297626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1268/2499] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:27.315488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:27.317094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:27.424523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c05c2-37dc-41f0-b3cf-69e67383e7e4 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:27.432898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:27.433594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:27.433892 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:27.434164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:27.443795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:27.446070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:27.458067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1232/2500] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:30.447461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d8d8-6820-454c-b034-df326220b372 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:30.449192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d8d8-6820-454c-b034-df326220b372 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:30.450146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d8d8-6820-454c-b034-df326220b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:30.482892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7731d8d8-6820-454c-b034-df326220b372 None None] MFA Rules not processed for user `d66dde93e0f14ce799c9c6294f216781`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:30.484692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7731d8d8-6820-454c-b034-df326220b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:30.496545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1269/2501] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:31.065522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:31.087685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dfa5-134a-4a86-a405-a6c90b2fecd0 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:31.100324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:31.101246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:31.101699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:31.102152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:31.105441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:31.119561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:31.122854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:31.147302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1233/2502] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:34.625681 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:34.626259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:34.626259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:34.648058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] MFA Rules not processed for user `8bd1061f505d4249ae14c7cbf68a950a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:34.649312 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:34.715846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1270/2503] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.141282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=72 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:44.709229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=66 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:04.631730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:04.633873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:04.655801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c36a-d445-43fe-9f15-0a0ef6dd9360 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:04.661420 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:04.662667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:04.663143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:04.663615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] PATH_INFO: `/v3/users/5c7220e14fcd40c9a78bbcef1a1db5d9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:04.665259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7220e14fcd40c9a78bbcef1a1db5d9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:04.667669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:04.727329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7220e14fcd40c9a78bbcef1a1db5d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:04.735575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7220e14fcd40c9a78bbcef1a1db5d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:04.750920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7220e14fcd40c9a78bbcef1a1db5d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:04.751678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7220e14fcd40c9a78bbcef1a1db5d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:04.752284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ea9d4bc-a1fc-4f4b-aecc-a16348958ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7220e14fcd40c9a78bbcef1a1db5d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:04.768989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1234/2504] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:04 2022] DELETE /identity/v3/users/5c7220e14fcd40c9a78bbcef1a1db5d9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:04.979726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:04.996858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3b79-4f01-46fd-a658-bba036143ed4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:05.082693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:05.083479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.083765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.084051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] PATH_INFO: `/v3/projects/56792f3625e2457bbe9aa0dfa9763028` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.085380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56792f3625e2457bbe9aa0dfa9763028)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:05.087381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:05.152310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56792f3625e2457bbe9aa0dfa9763028', 'request_id': 'req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.160082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56792f3625e2457bbe9aa0dfa9763028', 'request_id': 'req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.170167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56792f3625e2457bbe9aa0dfa9763028', 'request_id': 'req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.184477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] The token cache is being invalidate because project 56792f3625e2457bbe9aa0dfa9763028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:05.186029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.186882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.188550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1271/2505] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:04 2022] DELETE /identity/v3/projects/56792f3625e2457bbe9aa0dfa9763028 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:06.713114 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458e1b-9539-47c2-a0dc-78b188340753 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.713656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458e1b-9539-47c2-a0dc-78b188340753 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.713947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e458e1b-9539-47c2-a0dc-78b188340753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.756374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7e458e1b-9539-47c2-a0dc-78b188340753 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:06.757834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e458e1b-9539-47c2-a0dc-78b188340753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:06.824732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1235/2506] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:06.884343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.885869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:06.966383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffafd3-eee7-4d43-b7d1-f3cb842b8074 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.971651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.972365 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.972702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.973023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.982828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.984456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c2f9-1d60-49c8-b27c-b48aaaf67d18 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.994885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1272/2507] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:46:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:46:16.816869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:16.991647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:17.083848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358ca5d-b91a-439a-b91e-5f47f0014840 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.084926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358ca5d-b91a-439a-b91e-5f47f0014840 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.085695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358ca5d-b91a-439a-b91e-5f47f0014840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.123094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f358ca5d-b91a-439a-b91e-5f47f0014840 None None] MFA Rules not processed for user `6c7e0694c5ab46a4adef7a1f7b1fed4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:17.125338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f358ca5d-b91a-439a-b91e-5f47f0014840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:17.235872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1236/2508] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:23.712980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.713549 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.713828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.716403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1273/2509] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:46:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:23.725493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35bc8c-0ba8-4be4-9465-f94b4fb6dda0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.726758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35bc8c-0ba8-4be4-9465-f94b4fb6dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.727287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35bc8c-0ba8-4be4-9465-f94b4fb6dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.750801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6f35bc8c-0ba8-4be4-9465-f94b4fb6dda0 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:23.752026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f35bc8c-0ba8-4be4-9465-f94b4fb6dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.840313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1237/2510] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:23.854632 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.856685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.954958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a515568-2275-43ca-987e-88ac1640a57c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.962536 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.963366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.963852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.964284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.978900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.982032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1d8d7-df8d-4067-9127-4af862b47b15 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.014938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1274/2511] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:24.138753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.138753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.138753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.140077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1238/2512] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:46:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:24.148921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730b34-1b54-4103-b042-342c4005216e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.149433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730b34-1b54-4103-b042-342c4005216e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.149903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730b34-1b54-4103-b042-342c4005216e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.171492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-39730b34-1b54-4103-b042-342c4005216e None None] MFA Rules not processed for user `1747a2eb4850417d8d02a2b0a8d0cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:24.172725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-39730b34-1b54-4103-b042-342c4005216e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:24.268288 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1275/2513] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:24.280926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.283169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:24.297516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a264d-71b2-477f-9103-d83a8cffe314 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.301611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.302278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.302573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.302848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.312823 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.314601 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537e624-057d-406f-ab4d-6ce2ef4b83a2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.324122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1239/2514] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:24.412670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.412956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.413235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.414677 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1276/2515] 10.222.0.77 () {54 vars in 921 bytes} [Mon Jul 18 16:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:24.421516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562b10-e6a3-4247-8a25-e0d3f16b8cb3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.421812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562b10-e6a3-4247-8a25-e0d3f16b8cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.422087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4562b10-e6a3-4247-8a25-e0d3f16b8cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.441991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d4562b10-e6a3-4247-8a25-e0d3f16b8cb3 None None] MFA Rules not processed for user `bd1cd1cd83fa43cab0dcbd1698c715fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:24.443257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d4562b10-e6a3-4247-8a25-e0d3f16b8cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:24.452905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1240/2516] 10.222.0.77 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:24.463714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.465498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:24.479732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d89a9-f35c-4ccd-abd3-5bad8476d455 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.483943 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.484659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.484972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.485386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.498060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.499538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.512131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1277/2517] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:26.574195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bacc-8542-404f-bdcf-41c4d0d6a30b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:26.574863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bacc-8542-404f-bdcf-41c4d0d6a30b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:26.575242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bacc-8542-404f-bdcf-41c4d0d6a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:26.609024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fd02bacc-8542-404f-bdcf-41c4d0d6a30b None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:26.611105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fd02bacc-8542-404f-bdcf-41c4d0d6a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:26.625904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1241/2518] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:26.647261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:26.667693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bfa8-f902-49db-9b5d-b06839b64ca2 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:26.681550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:26.683400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:26.684069 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:26.684736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:26.702770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:26.706040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:26.720375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1278/2519] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:46:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:46:36.598824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=68 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:36.716009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=61 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:47.038796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.039619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.039788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.066233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:47.068206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.082328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1242/2520] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:47.102802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.116447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7347a4-94ef-4ab2-a9a0-d68cb2fd174c service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.123075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.124008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.124439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.124806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.136469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.138133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:47.148249 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1279/2521] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:46:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:46:57.057840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:57.144476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:04.537802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.553611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0f676-2582-41f3-bd5e-15c2d6c76f42 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.557828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.558598 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.558948 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.559265 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.561343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:04.631482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.632954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:04.644657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1243/2522] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:08.593891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:08.595856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.702614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:08.706166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.713301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae195d5e-4c52-4bf0-95eb-90560770f496 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:08.718901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:08.720457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:08.720457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:08.720457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:08.722516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.726206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fe9bf-ce91-4bc3-917f-1c2f5122c7e2 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:08.735029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:08.736918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:08.738311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:08.739434 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] PATH_INFO: `/v3/users/8bd1061f505d4249ae14c7cbf68a950a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:08.742241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd1061f505d4249ae14c7cbf68a950a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:08.742930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:08.744593 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:08.746085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:08.761106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1280/2523] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:08.810987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd1061f505d4249ae14c7cbf68a950a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.819342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd1061f505d4249ae14c7cbf68a950a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.836313 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd1061f505d4249ae14c7cbf68a950a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.837546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd1061f505d4249ae14c7cbf68a950a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.838545 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26a6b881-e6a9-42d3-adf0-58e408fb6fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd1061f505d4249ae14c7cbf68a950a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.855242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1244/2524] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:08 2022] DELETE /identity/v3/users/8bd1061f505d4249ae14c7cbf68a950a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:09.083050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.105736 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80eca4-412d-46f5-b5d3-db4381a355a1 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.183780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.184764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.185052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.185334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] PATH_INFO: `/v3/projects/dbd9c6bf5b0c491299815d545c1a594e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.186737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd9c6bf5b0c491299815d545c1a594e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.189009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.251385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd9c6bf5b0c491299815d545c1a594e', 'request_id': 'req-67f88a2c-586d-456a-9f02-8d3f975a3f09'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.264006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd9c6bf5b0c491299815d545c1a594e', 'request_id': 'req-67f88a2c-586d-456a-9f02-8d3f975a3f09'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.275174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd9c6bf5b0c491299815d545c1a594e', 'request_id': 'req-67f88a2c-586d-456a-9f02-8d3f975a3f09'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.287783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:09.295430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] The token cache is being invalidate because project dbd9c6bf5b0c491299815d545c1a594e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:09.296100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.297404 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67f88a2c-586d-456a-9f02-8d3f975a3f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.302490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1281/2525] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:09 2022] DELETE /identity/v3/projects/dbd9c6bf5b0c491299815d545c1a594e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:09.305833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.306142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.306456 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.325627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:09.326896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.401999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1245/2526] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:09.412054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.412345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.412626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.433240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:09.434500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.443583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1282/2527] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:09.448497 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.460059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cf2d8-8b4d-47ab-98a7-56b48bd437a5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.465291 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.466031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.466314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.466647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.467416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:09.467724 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.469460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.471375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:09.490714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1246/2528] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:09.495542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.513024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9b08-bc64-45e1-9d9b-c098f5dc0ca6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.518379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.519281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.519567 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.519846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.520565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.522531 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.553168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1283/2529] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:09.560210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.577546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3c15-ec0b-4694-ba71-b04a9ad9bbf3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.649803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.650989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.651485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.652339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.652793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.654859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.681928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1247/2530] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:09.686627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.697350 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8b38-b99d-4389-a38f-c02cf96d14eb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.702867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.703604 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.703921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.704205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.704973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.706833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.719788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1284/2531] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:09.724920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.735970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5faf-dbda-4dc9-8cc4-5f25e5dd9264 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.742553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.743320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.743723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.744098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.744918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.747240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808a757-7990-49df-abbf-1871bf993b97 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.759178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1248/2532] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:09.764162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.779355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162977f-845e-461e-b0bf-87a3e77155bc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.787626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.789174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.789725 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.790240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] PATH_INFO: `/v3/projects/e7aa6a778ade43a8b243e3cd0913e674/users/1e3e3a7b66614919ba16ccb73abb7571/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.815773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7aa6a778ade43a8b243e3cd0913e674, user_id=1e3e3a7b66614919ba16ccb73abb7571, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.820452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.842060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1285/2533] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:09 2022] PUT /identity/v3/projects/e7aa6a778ade43a8b243e3cd0913e674/users/1e3e3a7b66614919ba16ccb73abb7571/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:09.853527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.853818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.854162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.892018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] MFA Rules not processed for user `1e3e3a7b66614919ba16ccb73abb7571`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:09.893281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.970254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1249/2534] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:09.976872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.040234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05ddf-18f4-48d9-aed2-b2b7387fc3c3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.045150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.045858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.046127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.046400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.047116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.049032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.076901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1286/2535] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:10.082427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.098333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e02b70-97a6-40b5-b53f-2aed5434cbb7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.164084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.164723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.165066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.165761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.165988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.167908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.176167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:10.192174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1250/2536] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:10.196359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.209700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6d9b-9415-4b8f-98ee-e63713bb6c15 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.214709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.215386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.215738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.215909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.216762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.218386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.230595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1287/2537] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:10.234911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.245811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e891551-30ee-4b9d-97ac-4431935547ed admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.250648 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.251549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.251831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.252109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] PATH_INFO: `/v3/projects/30a57f022c4746b0a22bce1500aa085a/users/fed9a71d69974ac68a62bde74c69d1a7/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.275751 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a57f022c4746b0a22bce1500aa085a, user_id=fed9a71d69974ac68a62bde74c69d1a7, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.281310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.301134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1251/2538] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:10 2022] PUT /identity/v3/projects/30a57f022c4746b0a22bce1500aa085a/users/fed9a71d69974ac68a62bde74c69d1a7/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:10.305700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.317595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a9ebe-9a54-4b61-a525-2bd10a3f78c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.383381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.384057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.384338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.384612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.385345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.387000 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e1375-b741-4023-93b4-a19897460c7a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.404004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1288/2539] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:10.408568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.421465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38cdb7-5923-4005-91d7-5b779290eaa7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.430799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.434176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.434738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.435208 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] PATH_INFO: `/v3/projects/30a57f022c4746b0a22bce1500aa085a/users/fed9a71d69974ac68a62bde74c69d1a7/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.440302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a57f022c4746b0a22bce1500aa085a, user_id=fed9a71d69974ac68a62bde74c69d1a7, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.445949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.461576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1252/2540] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:10 2022] PUT /identity/v3/projects/30a57f022c4746b0a22bce1500aa085a/users/fed9a71d69974ac68a62bde74c69d1a7/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:10.473198 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.473735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.474165 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.489581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:10.506609 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] MFA Rules not processed for user `fed9a71d69974ac68a62bde74c69d1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:10.507896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.611453 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1289/2541] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:10.624315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.626162 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.747423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c181072-31a0-47d1-ba80-a1cb63339263 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.754530 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.755705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.756416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.756896 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.774766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.777459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667544f-96f8-41d8-b84e-ead9e40de124 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.796632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1253/2542] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:11.239081 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.314757 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea191db2-4488-4a1a-8cf3-b7a7fd4da8d9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.320596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.321890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.322229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.322804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] PATH_INFO: `/v3/users/1e3e3a7b66614919ba16ccb73abb7571` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.324447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3e3a7b66614919ba16ccb73abb7571)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.326644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.375857 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3e3a7b66614919ba16ccb73abb7571'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.383887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3e3a7b66614919ba16ccb73abb7571'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.396482 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3e3a7b66614919ba16ccb73abb7571'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.404980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3e3a7b66614919ba16ccb73abb7571'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.405827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3e3a7b66614919ba16ccb73abb7571'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.415602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1290/2543] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:11 2022] DELETE /identity/v3/users/1e3e3a7b66614919ba16ccb73abb7571 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:11.420623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.420917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.421200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.439453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:11.441364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.501553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1254/2544] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:11.511973 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.513414 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.527037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:11.574778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c777886-e5c8-4b39-b893-8e3219d0cbc4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.578477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.579137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.579428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.579752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.590568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.592055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.604791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1291/2545] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:11.928245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.929763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.946847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fdbd7-f169-49f0-a4e3-56354819fb21 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.954827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.955760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.956339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.956800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] PATH_INFO: `/v3/projects/e7aa6a778ade43a8b243e3cd0913e674` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.959829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7aa6a778ade43a8b243e3cd0913e674)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.963919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.009147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=4,keystone:SELECT=94,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:12.032540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7aa6a778ade43a8b243e3cd0913e674', 'request_id': 'req-360c43f3-2130-49ee-89f1-0377084f1212'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.044305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7aa6a778ade43a8b243e3cd0913e674', 'request_id': 'req-360c43f3-2130-49ee-89f1-0377084f1212'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.052311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7aa6a778ade43a8b243e3cd0913e674', 'request_id': 'req-360c43f3-2130-49ee-89f1-0377084f1212'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.063218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] The token cache is being invalidate because project e7aa6a778ade43a8b243e3cd0913e674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:12.063518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.064193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.065826 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1255/2546] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:11 2022] DELETE /identity/v3/projects/e7aa6a778ade43a8b243e3cd0913e674 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:12.070098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.080574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e903978-f39c-409f-baf8-4c1c94c87693 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.144402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.145301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.145582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.145866 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] PATH_INFO: `/v3/users/fed9a71d69974ac68a62bde74c69d1a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.147191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed9a71d69974ac68a62bde74c69d1a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.149180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.212207 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed9a71d69974ac68a62bde74c69d1a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.221490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed9a71d69974ac68a62bde74c69d1a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.235240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed9a71d69974ac68a62bde74c69d1a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.243707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed9a71d69974ac68a62bde74c69d1a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.244444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632d26cd-54e9-438c-a399-a7f2ddaffdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed9a71d69974ac68a62bde74c69d1a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.251022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1292/2547] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:12 2022] DELETE /identity/v3/users/fed9a71d69974ac68a62bde74c69d1a7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:12.578915 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.580856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.598666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c43f3-2130-49ee-89f1-0377084f1212 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.671279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.672185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.672468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.672742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] PATH_INFO: `/v3/projects/30a57f022c4746b0a22bce1500aa085a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.674136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a57f022c4746b0a22bce1500aa085a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.676287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.740573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a57f022c4746b0a22bce1500aa085a', 'request_id': 'req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.751500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a57f022c4746b0a22bce1500aa085a', 'request_id': 'req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.763625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a57f022c4746b0a22bce1500aa085a', 'request_id': 'req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.778177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] The token cache is being invalidate because project 30a57f022c4746b0a22bce1500aa085a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:12.778490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.779183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ddce4022-f3bb-4ecb-96e7-7c5463b273e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.780786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1256/2548] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:12 2022] DELETE /identity/v3/projects/30a57f022c4746b0a22bce1500aa085a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:12.786311 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.786698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.787024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.807639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:12.809181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.844874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:12.882720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1293/2549] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:12.893558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.893917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.894167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.913588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:12.914858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.923428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1257/2550] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:12.928721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.939363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6132e-f190-41e5-9da3-c77f06d9ebfd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.946121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.946852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.947142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.947419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.948183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:12.948498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.950130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.965512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1294/2551] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:12.970546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.980941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d788a95-bb81-4305-9ef2-f46867e5946c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.986620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.988115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.988426 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.988705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.989441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.991464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.020802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1258/2552] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:13.026356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.039240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c3e0-5d3c-4ed8-b144-ae70f2cb7181 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.119005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.120146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.120615 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.121113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.122580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.126846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.158923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1295/2553] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:13.163325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.175509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c2aa-06b5-4bc3-98cd-9e6cdacd821c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.180666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.181408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.181729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.182163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.183479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.185920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.198147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1259/2554] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:13.202386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.213582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27af62-df65-47ee-8781-05edefccfbd6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.218686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.219541 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.219802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.220101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.221082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.223237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d60aad-e1b6-4a4b-af70-dab804613479 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.235543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1296/2555] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:13.240937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.251845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31597e7-6a5f-4863-933e-63cbab9cd590 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.257186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.258100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.258379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.258686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] PATH_INFO: `/v3/projects/831edea79a9a40dda103b5df27a3251b/users/ff75104199664c4c9499f923ead61e2e/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.267240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:13.276549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=831edea79a9a40dda103b5df27a3251b, user_id=ff75104199664c4c9499f923ead61e2e, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.279380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.296439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1260/2556] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:13 2022] PUT /identity/v3/projects/831edea79a9a40dda103b5df27a3251b/users/ff75104199664c4c9499f923ead61e2e/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:13.307986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.308752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.308752 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.337724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] MFA Rules not processed for user `ff75104199664c4c9499f923ead61e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:13.339010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.410221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1297/2557] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:13.416800 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.500372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3143c-3338-42d7-831f-45f6c2f0fc26 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.508325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.509588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.510083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.510635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.512011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.515825 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.553258 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1261/2558] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:13.558633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.569649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0905ac-0534-4b5a-9387-f529807ab3fb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.604770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:13.656589 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.657362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.657650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.657932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.658672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.661205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.683117 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1298/2559] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:13.688727 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.706965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac2ed-1b8a-452f-8602-6931018e1e65 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.715401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.716484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.717100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.717554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.718901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.721585 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.739862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1262/2560] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:13.744591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.755355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded66f3e-87a9-45e5-9f3d-55eb7e42a802 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.760305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.761231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.761517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.761794 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] PATH_INFO: `/v3/projects/f13fb5868a304a3eaab3f2326185b731/users/18305b09db694fe98e065cd38c312a6e/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.779701 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13fb5868a304a3eaab3f2326185b731, user_id=18305b09db694fe98e065cd38c312a6e, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.782429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.798040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1299/2561] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:13 2022] PUT /identity/v3/projects/f13fb5868a304a3eaab3f2326185b731/users/18305b09db694fe98e065cd38c312a6e/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:13.802076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.813125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269867f7-0031-441b-9770-88d448bd6b0d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.872615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.873281 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.873700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.873973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.874605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.876233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db325-1412-44e2-9eba-71743a522301 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.886926 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1263/2562] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:13.891602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.903483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3bf2a-bce0-41c1-b9c1-034e9ba10de6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.908509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.909953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.910233 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.910537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] PATH_INFO: `/v3/projects/f13fb5868a304a3eaab3f2326185b731/users/18305b09db694fe98e065cd38c312a6e/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.912694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13fb5868a304a3eaab3f2326185b731, user_id=18305b09db694fe98e065cd38c312a6e, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.915458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.929348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1300/2563] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:13 2022] PUT /identity/v3/projects/f13fb5868a304a3eaab3f2326185b731/users/18305b09db694fe98e065cd38c312a6e/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:13.947122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.947667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.948164 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.985861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] MFA Rules not processed for user `18305b09db694fe98e065cd38c312a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:13.987134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.040008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:14.069329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1264/2564] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:14.081894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.083742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.148808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcaed1-da95-4d56-b55d-de83b1ffa12d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.152560 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.153227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.153504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.153781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.163726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.165581 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.176219 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1301/2565] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:14.225071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.226490 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.238734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463ce3e-e17b-437a-b25b-012047f368cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.242488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.243143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.243413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.243695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.305160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.306547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02076e84-8407-4cf5-93fc-1f2f4b689952 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.316790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1265/2566] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:17.895816 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.975161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165851a-0b99-4ec9-8cc1-247b68b7f7d2 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.979697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.980584 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.981006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.981261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] PATH_INFO: `/v3/users/ff75104199664c4c9499f923ead61e2e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.982383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff75104199664c4c9499f923ead61e2e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.984366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:18.025300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:18.053478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff75104199664c4c9499f923ead61e2e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.062648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff75104199664c4c9499f923ead61e2e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.077216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff75104199664c4c9499f923ead61e2e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.088150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff75104199664c4c9499f923ead61e2e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.089178 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff75104199664c4c9499f923ead61e2e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.099896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1302/2567] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:17 2022] DELETE /identity/v3/users/ff75104199664c4c9499f923ead61e2e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:18.106682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.107257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.107686 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.138061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:18.140152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.209980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1266/2568] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:18.222305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:18.224066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.286674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c353c31-2a9e-41f8-9136-60038ccb5609 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:18.290763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:18.291426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.291712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.291991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.302964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:18.304426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:18.315187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1303/2569] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:18.673762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:18.675837 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.690065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c869c7-52da-4dca-bce6-60cbb0b7a6c4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:18.695803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:18.696682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.696958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.697535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] PATH_INFO: `/v3/projects/831edea79a9a40dda103b5df27a3251b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.698570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=831edea79a9a40dda103b5df27a3251b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:18.700551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:18.786275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831edea79a9a40dda103b5df27a3251b', 'request_id': 'req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.796957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831edea79a9a40dda103b5df27a3251b', 'request_id': 'req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.805243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831edea79a9a40dda103b5df27a3251b', 'request_id': 'req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.818089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] The token cache is being invalidate because project 831edea79a9a40dda103b5df27a3251b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:18.818391 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.819096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.820989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1267/2570] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:18 2022] DELETE /identity/v3/projects/831edea79a9a40dda103b5df27a3251b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:18.825125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:18.837074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62684755-ffb5-4339-9c92-dc6ffde802bd service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:18.918457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:18.919596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.919833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.920342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] PATH_INFO: `/v3/users/18305b09db694fe98e065cd38c312a6e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.921763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18305b09db694fe98e065cd38c312a6e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:18.924104 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:18.991430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18305b09db694fe98e065cd38c312a6e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.993518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:19.006927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18305b09db694fe98e065cd38c312a6e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.022232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18305b09db694fe98e065cd38c312a6e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.031780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18305b09db694fe98e065cd38c312a6e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.033161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-683e13c7-f5e5-408d-b1e4-221b95c44364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18305b09db694fe98e065cd38c312a6e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.043354 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1304/2571] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:18 2022] DELETE /identity/v3/users/18305b09db694fe98e065cd38c312a6e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:19.369128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.371130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:19.382700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:19.387092 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52d743-de03-466b-bcfa-6ba8c14ce0b0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.457885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.459005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.459392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.459747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] PATH_INFO: `/v3/projects/f13fb5868a304a3eaab3f2326185b731` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.461122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13fb5868a304a3eaab3f2326185b731)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.463390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.516774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13fb5868a304a3eaab3f2326185b731', 'request_id': 'req-1719a04d-9d6c-4a44-9d12-138e7c4718ae'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.526927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13fb5868a304a3eaab3f2326185b731', 'request_id': 'req-1719a04d-9d6c-4a44-9d12-138e7c4718ae'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.534024 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13fb5868a304a3eaab3f2326185b731', 'request_id': 'req-1719a04d-9d6c-4a44-9d12-138e7c4718ae'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.545321 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] The token cache is being invalidate because project f13fb5868a304a3eaab3f2326185b731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:19.545609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.546288 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1719a04d-9d6c-4a44-9d12-138e7c4718ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.547917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1268/2572] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:19 2022] DELETE /identity/v3/projects/f13fb5868a304a3eaab3f2326185b731 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:19.556138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.556648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.557148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.582176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:19.583538 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:19.676021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1305/2573] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:19.685591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82094523-5881-4010-9aef-d27d10262d46 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.685975 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82094523-5881-4010-9aef-d27d10262d46 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.686320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82094523-5881-4010-9aef-d27d10262d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.707293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-82094523-5881-4010-9aef-d27d10262d46 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:19.708543 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82094523-5881-4010-9aef-d27d10262d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:19.721438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1269/2574] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:19.726371 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.736760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af3689-5b88-42d6-a728-e5b5cf789cc9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.742173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.743072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.743409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.743805 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.744680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:19.745148 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.747026 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.760030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1306/2575] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:19.765712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82094523-5881-4010-9aef-d27d10262d46 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.782987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82094523-5881-4010-9aef-d27d10262d46 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.789968 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.790711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.790994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.791267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.791979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.794036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.826064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1270/2576] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:19 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:19.830875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.843498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60b747-6012-4895-b9d3-8367096e13a6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.917112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.918351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.918908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.919602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.920960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.924781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.953324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1307/2577] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:19 2022] POST /identity/v3/users => generated 281 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:19.959318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.972293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d1df-fad1-4eed-86dd-9f68b3c36599 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.977297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.978097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.978390 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.978741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.979482 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.981106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.992947 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1271/2578] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:19.998856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.010820 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d060dd-756d-4a8e-8b95-902f864432c4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.015824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.016444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.016781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.017125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.017865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.020550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126a34a-c80c-4322-a5e9-4dd73280a1ad admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.032826 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1308/2579] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:20.037570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.048824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba892-36a8-4742-9215-ae70e2bcadd0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.054128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.055145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.055485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.055829 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] PATH_INFO: `/v3/projects/9002382fb58040fa9f080b1a00978f20/users/640cb7b0c8eb4423ba5c03b14d4db9d8/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.079694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9002382fb58040fa9f080b1a00978f20, user_id=640cb7b0c8eb4423ba5c03b14d4db9d8, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.083304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.104010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1272/2580] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:20 2022] PUT /identity/v3/projects/9002382fb58040fa9f080b1a00978f20/users/640cb7b0c8eb4423ba5c03b14d4db9d8/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:20.115901 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.116359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.116682 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.148421 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] MFA Rules not processed for user `640cb7b0c8eb4423ba5c03b14d4db9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:20.149666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.171154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:20.234583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1309/2581] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:20.271710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.273547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.312473 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:20.353620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7021f-3260-4521-ac84-f75da57ae54d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.359933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.361055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.361573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.362089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.376753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.379436 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da47d-5d7b-4435-a9d7-de075f73972e service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.389859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1273/2582] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:20.405526 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.504720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab313d-fa13-41a7-837f-77f5cf20d44f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.512684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.514464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.514955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.515451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] PATH_INFO: `/v3/users/640cb7b0c8eb4423ba5c03b14d4db9d8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.517665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=640cb7b0c8eb4423ba5c03b14d4db9d8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.521346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.597441 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640cb7b0c8eb4423ba5c03b14d4db9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.606004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640cb7b0c8eb4423ba5c03b14d4db9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.625769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640cb7b0c8eb4423ba5c03b14d4db9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.637020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640cb7b0c8eb4423ba5c03b14d4db9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.637991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640cb7b0c8eb4423ba5c03b14d4db9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.647294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1310/2583] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:20 2022] DELETE /identity/v3/users/640cb7b0c8eb4423ba5c03b14d4db9d8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:20.652635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.652635 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.652918 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.671546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:20.672790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.745691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1274/2584] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:20.757501 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.759803 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.839822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:20.860317 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75882778-f985-4068-860e-4766b3686b5e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.865064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.865888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.866211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.866575 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.877188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.878846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6ae78-d857-4553-9236-81afbfc434c7 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.889976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1311/2585] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:21.287599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.290168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:21.310869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee10a00-ea10-4288-a164-9ac7dd2c47f0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.318206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.320172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.320599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.320989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] PATH_INFO: `/v3/projects/9002382fb58040fa9f080b1a00978f20` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.323017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9002382fb58040fa9f080b1a00978f20)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.327046 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.397691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9002382fb58040fa9f080b1a00978f20', 'request_id': 'req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.408306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9002382fb58040fa9f080b1a00978f20', 'request_id': 'req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.417156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9002382fb58040fa9f080b1a00978f20', 'request_id': 'req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.431472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] The token cache is being invalidate because project 9002382fb58040fa9f080b1a00978f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:21.431851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.432763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b62e0e4b-d1ab-4fa8-bf56-4a8df365991a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.434566 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1275/2586] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:21 2022] DELETE /identity/v3/projects/9002382fb58040fa9f080b1a00978f20 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:21.439990 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.440745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.440745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.462172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:21.464077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:21.563724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1312/2587] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:21.573677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.574131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.574547 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.597714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:21.599066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:21.607857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1276/2588] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:21.612959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.625377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233abd25-0926-4e0a-9259-4e2d9b5f2f3d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.631191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.631890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.632224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.632512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.633338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:21.633678 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.635476 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.652971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1313/2589] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:21.659459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.670862 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf47ce-2917-4055-b106-99072f3394ab None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.676299 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.677018 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.677302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.677581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.678339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.680485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.712963 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1277/2590] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:21 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:21.718144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.730505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc925e44-1fde-4850-8a9b-d965888ae5f3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.801237 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.801941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.802260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.802574 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.803329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.805436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.829957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1314/2591] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:21 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:21.837064 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.854226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeda3cb-450b-47a1-8bb8-48a9940f027f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.861569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.863148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.863604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.864068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.865449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.868278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.887725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1278/2592] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:21.892040 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.903737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a689f-814b-4e54-a4d0-bbaaa1b9d1ad admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.908771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.909460 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.909776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.910128 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.911001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.912892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f3f3-4bd9-43b7-bd63-bedadc7d171c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.927175 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1315/2593] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:21.931814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.943990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be42c-109e-4a13-83c8-76af9c66395e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.949730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.950736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.951015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.951292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] PATH_INFO: `/v3/projects/b99e2d10d87a49329abd13ccd6ad428e/users/ca9788ca9e9042b8b1ac962d9e325f55/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.971726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99e2d10d87a49329abd13ccd6ad428e, user_id=ca9788ca9e9042b8b1ac962d9e325f55, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.974676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.977599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:21.994200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1279/2594] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:21 2022] PUT /identity/v3/projects/b99e2d10d87a49329abd13ccd6ad428e/users/ca9788ca9e9042b8b1ac962d9e325f55/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:22.007086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.007423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.007743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.039999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] MFA Rules not processed for user `ca9788ca9e9042b8b1ac962d9e325f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:22.042335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:22.120760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:22.157362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1316/2595] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:22.170708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.173486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:22.240270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62220240-05fc-4398-ba30-115fe4919449 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.244618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.245341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.245669 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.245995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.257051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.258834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.269183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1280/2596] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:30.914382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:30.916290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.037212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623e12-de0c-49e1-88a9-a1bda3cb6c4e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.046286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.047475 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.047979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.048478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.052173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.070783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.073814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.095461 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1317/2597] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:32.264553 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.805363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:35.807076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:35.872168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0db5-3531-4e68-8ad4-0857768e671e service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:35.876077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:35.876787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:35.877070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:35.877349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:35.879875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:35.962668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:35.964170 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:35.975131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1281/2598] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:40.205002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:40.206925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:40.228018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0187-043b-4623-b58f-6475ea80e755 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:40.233750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:40.234957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.235435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.236046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] PATH_INFO: `/v3/users/4909d4aef302404290adcd3268e53c1b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.238193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4909d4aef302404290adcd3268e53c1b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:40.240930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:40.310765 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4909d4aef302404290adcd3268e53c1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.322972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4909d4aef302404290adcd3268e53c1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.338166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4909d4aef302404290adcd3268e53c1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.349470 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4909d4aef302404290adcd3268e53c1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.350657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c0b1dcce-a77f-450e-a285-0ec05d29170c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4909d4aef302404290adcd3268e53c1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.362329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1318/2599] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:40 2022] DELETE /identity/v3/users/4909d4aef302404290adcd3268e53c1b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:40.589359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:40.608887 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0c1aa-d042-4036-80b5-793d221210a6 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:40.691683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:40.692630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.692953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.693270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] PATH_INFO: `/v3/projects/26a41505f0ce4a66a02a5d954da278cb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.694755 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a41505f0ce4a66a02a5d954da278cb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:40.696778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:40.754681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a41505f0ce4a66a02a5d954da278cb', 'request_id': 'req-39bb5ba5-1bca-4174-80f1-1b515750674d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.760467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.767237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a41505f0ce4a66a02a5d954da278cb', 'request_id': 'req-39bb5ba5-1bca-4174-80f1-1b515750674d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.778587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a41505f0ce4a66a02a5d954da278cb', 'request_id': 'req-39bb5ba5-1bca-4174-80f1-1b515750674d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.793441 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] The token cache is being invalidate because project 26a41505f0ce4a66a02a5d954da278cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:40.793929 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.794781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39bb5ba5-1bca-4174-80f1-1b515750674d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:40.797105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1282/2600] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:40 2022] DELETE /identity/v3/projects/26a41505f0ce4a66a02a5d954da278cb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:40.803451 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.804308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.804651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.828512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:40.830027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:40.875050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.930448 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1319/2601] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:40.938746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.939250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.939752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.975990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:40.978633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:40.993942 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1283/2602] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:41.000599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.013703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54184a34-8f3c-48fd-b765-ddf09cd2bdd3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.022483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.023721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.024211 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.024702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.026098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:41.026704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.029800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.058648 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1320/2603] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:41.064955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.084033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1f0d7-df08-4be1-aaeb-66093966167b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.093236 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.094460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.094980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.095532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.096938 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.100749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.142964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1284/2604] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:41 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:41.146867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.162732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99db83-9557-4c0f-8b5a-0f79b1d442f4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.225368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.226582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.226914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.227293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.228093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.230180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.255598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1321/2605] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:41 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:41.260102 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.272346 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220847b2-5a39-4a10-bbcb-37bd0aa6651a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.277332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.278172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.278597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.278774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.279494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.281210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.293293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1285/2606] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:41.298345 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.309134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa057d7b-60c3-476b-a5e8-15d73cf7d79c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.314471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.315717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.316119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.316368 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.317111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.318786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecc4bb-05d1-40c6-9b3d-1ed0f5b14d16 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.329599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1322/2607] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:41.334576 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.345005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98907e86-46ae-46ea-b9ad-7da0fc8d4505 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.350215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.351165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.351439 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.351709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] PATH_INFO: `/v3/projects/00d53375a6d3433987b24c1c6c4c68d1/users/d1a02c8baa3141f0ad932f2a577000c6/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.369771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d53375a6d3433987b24c1c6c4c68d1, user_id=d1a02c8baa3141f0ad932f2a577000c6, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.372513 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.389048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1286/2608] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:41 2022] PUT /identity/v3/projects/00d53375a6d3433987b24c1c6c4c68d1/users/d1a02c8baa3141f0ad932f2a577000c6/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:41.399858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.400119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.400430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.441440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] MFA Rules not processed for user `d1a02c8baa3141f0ad932f2a577000c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:41.443778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:41.542228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1323/2609] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:41.554145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:41.556332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:41.690909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebd854-92f1-4577-b23f-89f3c9761a63 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:41.697674 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:41.698343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:41.698657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:41.698932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:41.708480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:41.710167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66513c-bdc9-47a8-bb09-a96e569e295f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:41.720627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1287/2610] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:50.404303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:50.447841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:50.478234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edf696-4b76-44db-a21b-a18403f19625 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:50.483245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:50.484144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:50.484435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:50.484718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] PATH_INFO: `/v3/users/d1a02c8baa3141f0ad932f2a577000c6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:50.486024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a02c8baa3141f0ad932f2a577000c6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:50.488024 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:50.548089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a02c8baa3141f0ad932f2a577000c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:50.557306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a02c8baa3141f0ad932f2a577000c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:50.571287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a02c8baa3141f0ad932f2a577000c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:50.580580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a02c8baa3141f0ad932f2a577000c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:50.582181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a02c8baa3141f0ad932f2a577000c6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:50.590606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1324/2611] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:50 2022] DELETE /identity/v3/users/d1a02c8baa3141f0ad932f2a577000c6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:50.597856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:50.598954 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:50.599189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:50.621378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:50.622676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:50.658123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:50.697998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1288/2612] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:50.711202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:50.714030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:50.788728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28388-ef4e-4779-906a-4b2a97520acb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:50.792719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:50.793399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:50.793685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:50.793965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:50.804571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:50.806050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c6495-4c40-4ac5-a2cd-1e96b6b5e010 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:50.817627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1325/2613] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:51.000171 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.003349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.016636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf58aa6-bab2-4e70-9df7-b732fecc64c0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.024789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.025866 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.026691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.027218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] PATH_INFO: `/v3/users/ca9788ca9e9042b8b1ac962d9e325f55` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.029054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9788ca9e9042b8b1ac962d9e325f55)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.031546 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.097001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9788ca9e9042b8b1ac962d9e325f55'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.107152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9788ca9e9042b8b1ac962d9e325f55'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.125407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9788ca9e9042b8b1ac962d9e325f55'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.126961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9788ca9e9042b8b1ac962d9e325f55'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.128640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9788ca9e9042b8b1ac962d9e325f55'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.151692 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1289/2614] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:50 2022] DELETE /identity/v3/users/ca9788ca9e9042b8b1ac962d9e325f55 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:51.160007 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.160377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.161290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.189167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:51.190195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.191515 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.194106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.272159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:51.290772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1326/2615] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:51.294864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0e01a-ecc8-4f0c-81c2-e358b8681fac admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.298009 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.300008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.301647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.303078 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.303481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.303863 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] PATH_INFO: `/v3/projects/00d53375a6d3433987b24c1c6c4c68d1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.305339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d53375a6d3433987b24c1c6c4c68d1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.308160 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.371360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d53375a6d3433987b24c1c6c4c68d1', 'request_id': 'req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.374787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:51.379279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47be1-b9b3-4519-b64e-fbb08e453855 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.383983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d53375a6d3433987b24c1c6c4c68d1', 'request_id': 'req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.393194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d53375a6d3433987b24c1c6c4c68d1', 'request_id': 'req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.405603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] The token cache is being invalidate because project 00d53375a6d3433987b24c1c6c4c68d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:51.405905 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.406589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c3b5ffc4-3fab-48d7-8e7d-fe47ecd0cfcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:51.408468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1290/2616] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:51 2022] DELETE /identity/v3/projects/00d53375a6d3433987b24c1c6c4c68d1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:51.413506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3e5c-37eb-403b-97fd-1fd7f90e50ff None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.413793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3e5c-37eb-403b-97fd-1fd7f90e50ff None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.414076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3e5c-37eb-403b-97fd-1fd7f90e50ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.438283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1f2d3e5c-37eb-403b-97fd-1fd7f90e50ff None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:51.440583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f2d3e5c-37eb-403b-97fd-1fd7f90e50ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.448018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.448791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.449191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.449474 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.451621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.518093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1291/2617] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:51.527107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.527412 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.527626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.528195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.529106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.539950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1327/2618] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:51.550042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:51.551315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.559662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1292/2619] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:51.564257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.576231 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786113f-cc2f-46c6-ac8b-b819440f0d66 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.582532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.583262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.583544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.583828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.584590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:51.584922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.591890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.604842 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1328/2620] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:51.611217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.626521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894fcb8-0e54-495d-9269-3e9b6bc6d086 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.637435 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.638130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.638457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.639151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.639503 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.641643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.681756 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1293/2621] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:51.683048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.695332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9026-6367-4c61-a542-66daae865d32 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.758234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:51.775635 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.776442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.776930 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.777208 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.777927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.780136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.810284 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1329/2622] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:51.811589 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.822946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec43a2-75dd-4923-9f1f-a66807ee9f3d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.828047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.828816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.829094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.829368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.830117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.832384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.844393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1294/2623] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:51.848997 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.862852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bc77b-cd84-44c7-bbc3-2ef467351ff7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.868089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.868789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.869077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.869374 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.870130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.871915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.884853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1330/2624] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:51.889377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.902442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48cc8e-06ac-41f1-b09c-c5b03b689fe7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.908518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:51.909596 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.910009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.910511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] PATH_INFO: `/v3/projects/4d07558a9b99489c978718774a6ee7a0/users/be3ad6ebf45f4f4bbed394f4b2688f30/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.932214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d07558a9b99489c978718774a6ee7a0, user_id=be3ad6ebf45f4f4bbed394f4b2688f30, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:51.935176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1118d5-6ab6-4ca8-a420-607e38e764be admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:51.953196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:51.955541 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1295/2625] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:51 2022] PUT /identity/v3/projects/4d07558a9b99489c978718774a6ee7a0/users/be3ad6ebf45f4f4bbed394f4b2688f30/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:51.958585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:51.965956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.966276 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.966707 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.988492 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b64e-0891-421b-bc14-19dd333b19fa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:51.997165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] MFA Rules not processed for user `be3ad6ebf45f4f4bbed394f4b2688f30`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:51.998621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.066843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.067731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.068020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.068299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] PATH_INFO: `/v3/projects/b99e2d10d87a49329abd13ccd6ad428e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.069540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99e2d10d87a49329abd13ccd6ad428e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.071550 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.072025 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1296/2626] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:52.076633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.083733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:52.091001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985773-3b2e-4a6b-a259-813da0bd8ab1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.097300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.098884 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.099166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.099438 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.100146 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.102096 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.143872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1297/2627] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:52.145451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.151049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99e2d10d87a49329abd13ccd6ad428e', 'request_id': 'req-7cc4b161-b9a6-4826-9f45-92a29df95590'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:52.159299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99e2d10d87a49329abd13ccd6ad428e', 'request_id': 'req-7cc4b161-b9a6-4826-9f45-92a29df95590'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:52.168890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99e2d10d87a49329abd13ccd6ad428e', 'request_id': 'req-7cc4b161-b9a6-4826-9f45-92a29df95590'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:52.200775 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] The token cache is being invalidate because project b99e2d10d87a49329abd13ccd6ad428e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:52.201086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:52.201666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7cc4b161-b9a6-4826-9f45-92a29df95590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:52.203728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1331/2628] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:51 2022] DELETE /identity/v3/projects/b99e2d10d87a49329abd13ccd6ad428e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:52.209230 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b0861-ea03-4f68-bc71-3f191966b48a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.209544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b0861-ea03-4f68-bc71-3f191966b48a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.209824 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b0861-ea03-4f68-bc71-3f191966b48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.228641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-129b0861-ea03-4f68-bc71-3f191966b48a None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:52.229884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-129b0861-ea03-4f68-bc71-3f191966b48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.247355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1db885-fc36-4aa8-971b-0a4f50f24835 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.255112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.256217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.256638 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.257081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.258347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.261632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.293501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1298/2629] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:52.294721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.296846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.303928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:52.311022 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4db25-e4c2-4b50-8986-15a8e410b7db admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.316494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.317254 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.317555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.317841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.324981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1332/2630] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:52.325272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.325272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.328277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.329105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.329642 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.332010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1299/2631] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:52.336624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.348142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa1493-638b-463e-bc34-c06e08f4607c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.353166 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.353859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.354150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.354459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.355218 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.356922 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.361316 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:52.364067 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.370033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1300/2632] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:52.374789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.382527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1333/2633] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:52.384399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.389614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9c95-cec3-47e2-abf6-eb8c06571eb4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.395429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.395972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6efab-62eb-43ce-b11c-75fa87d5f57f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.396470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.396842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.397278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] PATH_INFO: `/v3/projects/42e2f3374f694b12a72ed823c41923f8/users/bd3fb24bc3b3412b959edeb8014f13e8/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.402147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.402976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.403381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.403767 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.404623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:52.405031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.406817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.418380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e2f3374f694b12a72ed823c41923f8, user_id=bd3fb24bc3b3412b959edeb8014f13e8, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.422256 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c563d-c83a-42e4-9c68-f09686731b5a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.424235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1334/2634] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:52.431307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.440593 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1301/2635] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:52 2022] PUT /identity/v3/projects/42e2f3374f694b12a72ed823c41923f8/users/bd3fb24bc3b3412b959edeb8014f13e8/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:52.445562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba984a-2846-4630-998a-b26b3f1770ae admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.455041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.455422 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.456034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.495505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] MFA Rules not processed for user `bd3fb24bc3b3412b959edeb8014f13e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:52.496952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.529028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.530099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.530607 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.530991 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.531931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.535355 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.585246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1335/2636] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:52.590433 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.597765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1302/2637] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:52.602305 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.624246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f2336-1ac9-497c-9307-403f02bfff85 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.638182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:52.673386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116aab4-d0d0-449d-9673-ae913f6ce48a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.679679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.680369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.680651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.680923 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.681637 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.683668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.704587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.705852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.706314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.706910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.708139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.710236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1336/2638] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:52.713763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.714703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.726656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c6ea-ecaf-4f56-96b9-87c24c412e17 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.732382 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.733162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.733544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.734030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.734646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.736269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.748449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1303/2639] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:52.752465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1337/2640] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:52.754394 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.755631 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.770922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba78c9-a78c-40cc-9f21-c515ef1efb54 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.820791 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:52.834990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e1b4-906e-4dff-991d-e471b12f5298 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.840951 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.841523 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.841734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.841913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.842182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.842607 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.842952 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.843128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.843782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.844706 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.844964 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.847007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.860193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1304/2641] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:52.865326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.870920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1338/2642] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:52.876074 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab18d0-864f-4b4e-b3b7-2d7025bf4001 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.878257 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.881814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.882769 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.883053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.883329 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] PATH_INFO: `/v3/projects/7ff25285cbea432ebd3722887bdc2db4/users/46438e2022b2496a9506c18f4591e14e/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.894929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da836b-12c0-495e-916f-891fad738256 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.901334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.901677 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff25285cbea432ebd3722887bdc2db4, user_id=46438e2022b2496a9506c18f4591e14e, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.902353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.902788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.903163 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.904113 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.904296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545723-43a3-4ca8-9519-d3e8ff0470b1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.906058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.924030 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1305/2644] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:52 2022] PUT /identity/v3/projects/7ff25285cbea432ebd3722887bdc2db4/users/46438e2022b2496a9506c18f4591e14e/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:52.924260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1339/2644] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:52.931167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.944262 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.944729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.945174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.949813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c021-03f1-42af-bdac-4859272d686e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.984932 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] MFA Rules not processed for user `46438e2022b2496a9506c18f4591e14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:52.986116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.025986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.027573 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.028065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.028554 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] PATH_INFO: `/v3/projects/376230b997ac4a23839592276564c51b/users/5c5c9567d8d9471ab4c6aaaeaeccaced/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.055134 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376230b997ac4a23839592276564c51b, user_id=5c5c9567d8d9471ab4c6aaaeaeccaced, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.058426 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.076999 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1340/2645] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:52 2022] PUT /identity/v3/projects/376230b997ac4a23839592276564c51b/users/5c5c9567d8d9471ab4c6aaaeaeccaced/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:53.082121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.091220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1306/2646] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:53.100817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.102801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.108562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:53.164075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810004-48d7-40f1-9abe-2b1c3c15b445 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.169180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.169888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.170172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.170738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.171511 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.173274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.185437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1341/2647] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:53.189945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.195194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac3464-37e3-4a31-84d7-48f78303003d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.202294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099526-17ce-490e-8c6a-aa6e54020cf8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.207745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.208666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.208945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.209222 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] PATH_INFO: `/v3/projects/376230b997ac4a23839592276564c51b/users/5c5c9567d8d9471ab4c6aaaeaeccaced/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.211430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376230b997ac4a23839592276564c51b, user_id=5c5c9567d8d9471ab4c6aaaeaeccaced, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.214202 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38939de-20eb-455c-84dd-40a8e9a4cf71 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.227120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1342/2648] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:53 2022] PUT /identity/v3/projects/376230b997ac4a23839592276564c51b/users/5c5c9567d8d9471ab4c6aaaeaeccaced/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:53.238896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.239097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.239372 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.264734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.265916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.266507 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.267021 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.271523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] MFA Rules not processed for user `5c5c9567d8d9471ab4c6aaaeaeccaced`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:53.272933 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.293155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:53.345766 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.347807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.358623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1307/2649] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:53.363279 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1343/2650] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:53.375532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.377991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.386056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.394003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47fbb-23a0-45b5-b837-550184e14660 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.456902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.457885 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.458293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.458732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.473666 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.475312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ecb8-ab7e-4c86-9ac4-da770e24655d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.475713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9852e-5da5-4077-8a1d-087ffbe38f6f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.480447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.481337 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.481626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.481903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] PATH_INFO: `/v3/users/46438e2022b2496a9506c18f4591e14e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.483399 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46438e2022b2496a9506c18f4591e14e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.485369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.497017 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1308/2651] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:53.550986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46438e2022b2496a9506c18f4591e14e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:53.566221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46438e2022b2496a9506c18f4591e14e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:53.579493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46438e2022b2496a9506c18f4591e14e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:53.592686 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46438e2022b2496a9506c18f4591e14e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:53.593837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46438e2022b2496a9506c18f4591e14e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:53.604136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1344/2652] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:53 2022] DELETE /identity/v3/users/46438e2022b2496a9506c18f4591e14e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:53.609212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.609578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.609939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.639001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:53.641088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.661536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:53.734757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1309/2653] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:53.746818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.748693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.804089 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:53.834781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e89e5-d1ec-489e-b9cd-cc660032f963 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.838709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.839386 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.839671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.839954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.851369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.852934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.866061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1345/2654] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:54.267485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.270788 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.281353 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.283569 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.294083 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1be02-f495-462f-81f9-6172b6fdf6ab None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.304286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.305508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.305965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.306443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] PATH_INFO: `/v3/projects/7ff25285cbea432ebd3722887bdc2db4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.308537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff25285cbea432ebd3722887bdc2db4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.311936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.384340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e09bf-de67-4fb0-a88e-da97f593b8ad service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.389231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff25285cbea432ebd3722887bdc2db4', 'request_id': 'req-3ae7c083-8ea9-4b66-b5d6-731213c76715'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:54.401472 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff25285cbea432ebd3722887bdc2db4', 'request_id': 'req-3ae7c083-8ea9-4b66-b5d6-731213c76715'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:54.411555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff25285cbea432ebd3722887bdc2db4', 'request_id': 'req-3ae7c083-8ea9-4b66-b5d6-731213c76715'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:54.429338 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] The token cache is being invalidate because project 7ff25285cbea432ebd3722887bdc2db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:54.429845 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:54.430952 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3ae7c083-8ea9-4b66-b5d6-731213c76715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:54.433365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1310/2655] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:54 2022] DELETE /identity/v3/projects/7ff25285cbea432ebd3722887bdc2db4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:54.439353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4e-4f87-43ee-b2bc-0d3b21b7735f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.439684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4e-4f87-43ee-b2bc-0d3b21b7735f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.439981 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4e-4f87-43ee-b2bc-0d3b21b7735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.443170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.443841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.444138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.444449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.446401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.460871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-65a1fd4e-4f87-43ee-b2bc-0d3b21b7735f None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:54.462198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-65a1fd4e-4f87-43ee-b2bc-0d3b21b7735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.536086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.538939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.546118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1311/2656] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:54.555363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.555728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.556065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.557782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1346/2657] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:54.583956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:54.585939 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.599908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1312/2658] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:54.604938 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.620760 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a4b1d-4319-4d10-8932-5e5ef6ab0dc2 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.626957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.627650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.627932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.628216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.628955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:54.629262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.630941 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.654229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1347/2659] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:54.660011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.671848 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78d901-76ca-463f-bf12-2d0e99c4062e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.678782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.679498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.679815 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.680117 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.680897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.682917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.712283 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1313/2660] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:54 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:54.717660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.729205 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be687378-b56f-4a02-89a1-183633ac49dc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.776447 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:54.827341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.828041 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.828328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.828644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.829523 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.831839 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.863781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1348/2661] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:54 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:54.868542 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.881570 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda94543-b402-411e-863a-3f629003bab5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.889067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.889758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.890042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.890368 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.891151 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.893774 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.904463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:54.919551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1314/2662] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:54.925487 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.940671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582db9-4af6-4e22-897a-0e6de9da5386 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.947623 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.948299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.948580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.948853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.949594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:54.951366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31d59b-df52-4fec-b3e6-46c9c34ecdf7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:54.968138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1349/2663] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:54.972874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:54.989148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ff68-310e-43bd-88e7-d58cda8c97f2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:54.994223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:54.995282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:54.995849 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:54.996068 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] PATH_INFO: `/v3/projects/6343e3024ef0469da293401317da07f2/users/b54a5070a23e45a3975c0fa0f2bcc76e/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:55.013603 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6343e3024ef0469da293401317da07f2, user_id=b54a5070a23e45a3975c0fa0f2bcc76e, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:55.016652 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:55.035928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1315/2664] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:54 2022] PUT /identity/v3/projects/6343e3024ef0469da293401317da07f2/users/b54a5070a23e45a3975c0fa0f2bcc76e/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:55.041385 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:55.041694 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:55.041979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:55.063097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:55.064314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:55.131744 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1350/2665] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:55.142118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:55.144377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:55.217172 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89271c-8029-480e-996a-c95dfb2075ea admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:55.221077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:55.222296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:55.222627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:55.222899 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:55.232869 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:55.234324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bd447-ec0a-4611-ae94-c1c639689d0e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:55.245101 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1316/2666] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:59.073076 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.075412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.149672 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661706-5ab2-4220-a31f-b63f11709706 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.156906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.158641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.158641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.158641 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.162908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.270706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.272258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.284814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1351/2667] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:00.884015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.884582 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.884994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.945895 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] MFA Rules not processed for user `b54a5070a23e45a3975c0fa0f2bcc76e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:00.947169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:01.115902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1317/2668] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:01.120075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.132570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3bc9-d6e4-48ad-86eb-111fba14f0d3 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.137449 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.138169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.138490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.139293 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.139547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.141473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.145310 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:01.182793 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1352/2669] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:01.187088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.200858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bad368-07db-4957-ac71-c783a8fcbe69 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.322622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.323325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.323642 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.323936 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.324606 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.326840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.352902 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:01.365248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1318/2670] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:01.369496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.382020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c8ca-2b66-46c6-ad4c-4bde0d2dfeda admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.387555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.388430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.388720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.389008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.389870 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.391905 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.404900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1353/2671] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:01.409850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.422241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1227-4ad5-4e31-8912-79d3e4198976 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.430619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.432623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.433468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.434105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] PATH_INFO: `/v3/projects/1b1a074500614af99c90dee2ffb7dfbc/users/fcde37d1d66745f0938f158eb32e10c9/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.461924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1a074500614af99c90dee2ffb7dfbc, user_id=fcde37d1d66745f0938f158eb32e10c9, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.465440 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.483640 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1319/2672] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:01 2022] PUT /identity/v3/projects/1b1a074500614af99c90dee2ffb7dfbc/users/fcde37d1d66745f0938f158eb32e10c9/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:01.489070 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.501770 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aab4f1-9ea2-489e-976f-af03db8f7ad4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.577827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.578625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.578913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.579192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.580103 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.581911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.598268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1354/2673] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:01.599529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.611302 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511002f-60dd-4f18-ada7-c1efbe26420f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.618979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.620009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.620252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.620491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] PATH_INFO: `/v3/projects/1b1a074500614af99c90dee2ffb7dfbc/users/fcde37d1d66745f0938f158eb32e10c9/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.622986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1a074500614af99c90dee2ffb7dfbc, user_id=fcde37d1d66745f0938f158eb32e10c9, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.626050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed4bf-301d-44ad-bd45-6506a4602577 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.646318 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1320/2674] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:01 2022] PUT /identity/v3/projects/1b1a074500614af99c90dee2ffb7dfbc/users/fcde37d1d66745f0938f158eb32e10c9/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:02.141419 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.144665 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:02.159139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8098bd-04c2-4e0d-93c5-ce53f6c92978 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.237899 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.238831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.239116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.239393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] PATH_INFO: `/v3/users/be3ad6ebf45f4f4bbed394f4b2688f30` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.240948 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3ad6ebf45f4f4bbed394f4b2688f30)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.243045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.311953 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3ad6ebf45f4f4bbed394f4b2688f30'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:02.346811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3ad6ebf45f4f4bbed394f4b2688f30'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:02.383836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:02.392662 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3ad6ebf45f4f4bbed394f4b2688f30'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:02.411005 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3ad6ebf45f4f4bbed394f4b2688f30'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:02.411800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3ad6ebf45f4f4bbed394f4b2688f30'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:02.422893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1355/2675] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:02 2022] DELETE /identity/v3/users/be3ad6ebf45f4f4bbed394f4b2688f30 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:02.440075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.454739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.455421 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.508993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:02.510444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:02.586093 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1321/2676] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:02.597533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.599464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:02.698376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac92bc4-3319-438c-ac8a-665267e11a05 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.710468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.711186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.711473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.711750 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.728048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.731335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.744144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1356/2677] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:03.152604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:03.154552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:03.170277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762887c0-c28f-4d0c-8f1b-9d630a8d1dc6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:03.181418 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:03.182685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.184309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.184309 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] PATH_INFO: `/v3/projects/4d07558a9b99489c978718774a6ee7a0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.185847 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d07558a9b99489c978718774a6ee7a0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:03.187798 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:03.261548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d07558a9b99489c978718774a6ee7a0', 'request_id': 'req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.279016 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d07558a9b99489c978718774a6ee7a0', 'request_id': 'req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.293964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d07558a9b99489c978718774a6ee7a0', 'request_id': 'req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.312493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] The token cache is being invalidate because project 4d07558a9b99489c978718774a6ee7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:03.313047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.313647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.320512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1322/2678] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:03 2022] DELETE /identity/v3/projects/4d07558a9b99489c978718774a6ee7a0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:03.322402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:03.335792 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41725e4-f8ab-44b3-aced-3a6e1685bfaf service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:03.408078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:03.410289 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.410758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.411159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] PATH_INFO: `/v3/users/bd3fb24bc3b3412b959edeb8014f13e8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.412843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3fb24bc3b3412b959edeb8014f13e8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:03.415142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:03.487914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3fb24bc3b3412b959edeb8014f13e8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.498012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3fb24bc3b3412b959edeb8014f13e8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.514778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3fb24bc3b3412b959edeb8014f13e8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.527468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3fb24bc3b3412b959edeb8014f13e8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.528361 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3fb24bc3b3412b959edeb8014f13e8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:03.546817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1357/2679] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:03 2022] DELETE /identity/v3/users/bd3fb24bc3b3412b959edeb8014f13e8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:04.075050 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:04.076696 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:04.124813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383019d-bdbc-4c1d-82bf-b1ecdd0a3719 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:04.261555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:04.290927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:04.292498 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.292877 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.293347 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] PATH_INFO: `/v3/projects/42e2f3374f694b12a72ed823c41923f8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:04.296739 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e2f3374f694b12a72ed823c41923f8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:04.300100 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:04.423105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e2f3374f694b12a72ed823c41923f8', 'request_id': 'req-c87baf85-e0aa-49b0-909c-e99d451e5fa5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.439167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e2f3374f694b12a72ed823c41923f8', 'request_id': 'req-c87baf85-e0aa-49b0-909c-e99d451e5fa5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.458287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e2f3374f694b12a72ed823c41923f8', 'request_id': 'req-c87baf85-e0aa-49b0-909c-e99d451e5fa5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.484395 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] The token cache is being invalidate because project 42e2f3374f694b12a72ed823c41923f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:04.484984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.485953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.492946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1323/2680] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:04 2022] DELETE /identity/v3/projects/42e2f3374f694b12a72ed823c41923f8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:04.493183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:04.494863 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:04.545886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f594f-b083-4395-895e-0720b80816f4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:04.595112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:04.738314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:04.739664 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.740199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.740735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] PATH_INFO: `/v3/users/5c5c9567d8d9471ab4c6aaaeaeccaced` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:04.742557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5c9567d8d9471ab4c6aaaeaeccaced)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:04.746628 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:04.836718 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5c9567d8d9471ab4c6aaaeaeccaced'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.847188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5c9567d8d9471ab4c6aaaeaeccaced'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.863464 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5c9567d8d9471ab4c6aaaeaeccaced'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.872882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5c9567d8d9471ab4c6aaaeaeccaced'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.873585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ea8c940f-1bbe-416e-8439-39355811c1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5c9567d8d9471ab4c6aaaeaeccaced'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.886054 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1358/2681] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:04 2022] DELETE /identity/v3/users/5c5c9567d8d9471ab4c6aaaeaeccaced => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:05.485527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:05.487833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:05.525846 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87baf85-e0aa-49b0-909c-e99d451e5fa5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:05.791974 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:05.792878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:05.795650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:05.796054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] PATH_INFO: `/v3/projects/376230b997ac4a23839592276564c51b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.797916 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376230b997ac4a23839592276564c51b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:05.800155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:05.920717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376230b997ac4a23839592276564c51b', 'request_id': 'req-53d7bd97-2579-4340-ae35-3dc9a8c44a68'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.934797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376230b997ac4a23839592276564c51b', 'request_id': 'req-53d7bd97-2579-4340-ae35-3dc9a8c44a68'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.949630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376230b997ac4a23839592276564c51b', 'request_id': 'req-53d7bd97-2579-4340-ae35-3dc9a8c44a68'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.983852 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] The token cache is being invalidate because project 376230b997ac4a23839592276564c51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:05.984184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.986554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53d7bd97-2579-4340-ae35-3dc9a8c44a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.994723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1324/2682] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:05 2022] DELETE /identity/v3/projects/376230b997ac4a23839592276564c51b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:06.000035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.000626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.000626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.052624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:06.055830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.200336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1359/2683] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:06.214188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.214645 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.215055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.243444 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:06.285465 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:06.287297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.319212 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1325/2684] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:06.322347 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.343446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7c24d-6e72-4d39-8ce8-b690cb9c7d97 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.362236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.363329 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.363885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.364298 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.365614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:06.366056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.369098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.418836 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1360/2685] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:06.421672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.445181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721de15-5740-43db-9beb-2cebbe474242 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.465344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.469997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.476121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.476839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.480241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.482754 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.561118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1326/2686] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:06 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:06.566268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.582075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979a15f-e1ff-49b4-82e4-5ecacfde78e9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.624170 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:06.736131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.736928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.737281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.737645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.738527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.740921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.788852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1361/2687] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:06 2022] POST /identity/v3/users => generated 291 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:06.790451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.817335 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b53d6d-cd04-48a6-b7c4-03139b558107 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.828698 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.830273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.831616 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.833937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.842403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.850940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.890045 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1327/2688] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:06.891479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.913709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4a981-0ed3-4e59-bd46-c823be4ed62d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.922014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.922865 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.923221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.923568 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.924935 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.926797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e5389-5a4a-484c-a5f5-3f7ff067db8e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.946553 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1362/2689] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:06.948827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.980221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7c45-9eb6-4db2-8ce0-5bba2a4fe1d9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.991086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.992921 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.993673 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.994451 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] PATH_INFO: `/v3/projects/973fb24f9b4e4934a7895637e68d699e/users/9f660213244c40b4b9c64965443614dd/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.062526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=973fb24f9b4e4934a7895637e68d699e, user_id=9f660213244c40b4b9c64965443614dd, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.068627 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.118486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1328/2690] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:06 2022] PUT /identity/v3/projects/973fb24f9b4e4934a7895637e68d699e/users/9f660213244c40b4b9c64965443614dd/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:07.144903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.145837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.146169 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.200841 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] MFA Rules not processed for user `9f660213244c40b4b9c64965443614dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.202890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.354463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1363/2691] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:07.356657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.493737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd9d58-5d7e-464f-933e-0e91dfff2daa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.523682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.524467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.524957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.525449 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.527286 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.534066 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.640605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1329/2692] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/projects => generated 381 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:07.642463 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.656887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7d3d-3f4b-45cd-be12-8a7ed30800f3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.809772 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.811252 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.811986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.812260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.813590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.817424 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.860977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1364/2693] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:07.865447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.886737 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e39cc8-859a-4a76-8960-1ced5fbc64ea admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.903757 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.907634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.908226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.908987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.911827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.922174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.991259 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1330/2694] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:07.993471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:08.018377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6bfff-66e1-4592-91e3-da0934347d87 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.030356 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.034737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.035446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.036290 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] PATH_INFO: `/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf/users/f00073368acd40b58e02b9b5fa01b753/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.075375 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:08.082679 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd84d3bad1e444cbeba84c9c087e7cf, user_id=f00073368acd40b58e02b9b5fa01b753, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.085988 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.122754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1365/2695] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:07 2022] PUT /identity/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf/users/f00073368acd40b58e02b9b5fa01b753/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:08.127995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:08.158334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d430dbc-b19e-4949-932c-871e3a470131 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.266157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.267314 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.267665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.268142 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.269086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.271107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acae1f-e5f7-422f-a298-eeab9db5adac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.302537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1331/2696] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:08.309958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:08.346977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e7f7-50a5-461e-81e3-b1bf23ea6308 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.356155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.366445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.366445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.366445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] PATH_INFO: `/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf/users/f00073368acd40b58e02b9b5fa01b753/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.366445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd84d3bad1e444cbeba84c9c087e7cf, user_id=f00073368acd40b58e02b9b5fa01b753, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.375362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.434477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1366/2697] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:08 2022] PUT /identity/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf/users/f00073368acd40b58e02b9b5fa01b753/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:08.437350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.437721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.438077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.470691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:08.483384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] MFA Rules not processed for user `f00073368acd40b58e02b9b5fa01b753`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:08.486383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:08.618544 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1332/2698] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:08.621504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:08.623493 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:08.728048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29ddb-e89b-420a-83b6-4d12ae6be96d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.731949 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.732830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.734087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.734383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.747110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.748651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aa03b-3ebe-4b2c-984f-d4df50e9fb7c service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.769471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1367/2699] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:09.011964 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:09.027486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410d5f1-68d5-4f72-9556-da550f00db72 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:09.046133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:09.047486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.047983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.048457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.225719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:09.229269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:09.252198 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1333/2700] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:09.274608 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c9de-606c-41cd-beb8-a4b73b771a6d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.274909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c9de-606c-41cd-beb8-a4b73b771a6d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.275390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c9de-606c-41cd-beb8-a4b73b771a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.313613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b053c9de-606c-41cd-beb8-a4b73b771a6d None None] MFA Rules not processed for user `fcde37d1d66745f0938f158eb32e10c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:09.315610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b053c9de-606c-41cd-beb8-a4b73b771a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:09.429822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1368/2701] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:09.453130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:09.456056 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:09.573304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8621db-1a6e-4110-a2a4-7a6a1824e873 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:09.579382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2880530a3d43b482fb27ac23701293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:09.580155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.580442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.580802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.583971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:09.702312 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:09.719419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:09.721718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:09.741282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1334/2702] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:09.930533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.930845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.931098 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.952262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:09.953517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.026886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1369/2703] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:10.098591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.100371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.214735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4dd13-ebd9-49ad-b40a-08fcac099a19 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.221186 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.222501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.222836 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.223121 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.234051 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.235725 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84c4a0-bab0-4a62-b45f-f9eab189c56c service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.248231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1335/2704] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:48:12.368220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:12.386307 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:12.498647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d8d3-6b4a-44f8-afa1-ea5aa5c735d5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:12.506248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:12.507331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.508936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.509297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] PATH_INFO: `/v3/users/9f660213244c40b4b9c64965443614dd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.510887 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f660213244c40b4b9c64965443614dd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:12.513020 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:12.594845 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f660213244c40b4b9c64965443614dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:12.608712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f660213244c40b4b9c64965443614dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:12.628749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f660213244c40b4b9c64965443614dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:12.642045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f660213244c40b4b9c64965443614dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:12.644771 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f660213244c40b4b9c64965443614dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:12.653716 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1370/2705] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:12 2022] DELETE /identity/v3/users/9f660213244c40b4b9c64965443614dd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:12.659493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.660004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.660336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.693332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:12.695753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:12.854715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1336/2706] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:12.866146 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:12.869268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:12.993890 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db713c2-4f97-4b8a-b919-e85720029d31 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:13.001084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:13.002241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:13.002851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:13.003338 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:13.021685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:13.024945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:13.048328 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1371/2707] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:13.671626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:13.674363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:13.714733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab74077-4358-4b48-9089-d569adbb1767 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:13.724165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:13.725741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:13.726227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:13.726797 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] PATH_INFO: `/v3/projects/973fb24f9b4e4934a7895637e68d699e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:13.729229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973fb24f9b4e4934a7895637e68d699e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:13.733501 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:13.842781 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973fb24f9b4e4934a7895637e68d699e', 'request_id': 'req-ef7653a9-80af-4aa0-a668-34d9e549f1d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:13.859961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973fb24f9b4e4934a7895637e68d699e', 'request_id': 'req-ef7653a9-80af-4aa0-a668-34d9e549f1d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:13.872864 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973fb24f9b4e4934a7895637e68d699e', 'request_id': 'req-ef7653a9-80af-4aa0-a668-34d9e549f1d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:13.893475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] The token cache is being invalidate because project 973fb24f9b4e4934a7895637e68d699e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:13.894209 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:13.895647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:13.897803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1337/2708] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:13 2022] DELETE /identity/v3/projects/973fb24f9b4e4934a7895637e68d699e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:13.904258 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:13.920188 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8158d0-1f58-41e9-a2e1-342cc14869b1 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:13.985661 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:14.044700 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:14.046363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:14.046817 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:14.047235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] PATH_INFO: `/v3/users/f00073368acd40b58e02b9b5fa01b753` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:14.050132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00073368acd40b58e02b9b5fa01b753)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:14.053544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:14.173294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00073368acd40b58e02b9b5fa01b753'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:14.184377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00073368acd40b58e02b9b5fa01b753'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:14.201801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00073368acd40b58e02b9b5fa01b753'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:14.242097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00073368acd40b58e02b9b5fa01b753'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:14.243151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f5aa9ecc-01e0-4412-a5d5-fd45cbffc6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00073368acd40b58e02b9b5fa01b753'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:14.253888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1372/2709] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:13 2022] DELETE /identity/v3/users/f00073368acd40b58e02b9b5fa01b753 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:15.009468 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:15.011452 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:15.022950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:15.042089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7653a9-80af-4aa0-a668-34d9e549f1d6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:15.202012 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:15.205668 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:15.206403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:15.207076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] PATH_INFO: `/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:15.210029 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd84d3bad1e444cbeba84c9c087e7cf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:15.214447 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:15.430859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd84d3bad1e444cbeba84c9c087e7cf', 'request_id': 'req-1fdd0e87-b688-4e2b-999b-f0a89ef87799'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:15.448768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd84d3bad1e444cbeba84c9c087e7cf', 'request_id': 'req-1fdd0e87-b688-4e2b-999b-f0a89ef87799'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:15.470269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd84d3bad1e444cbeba84c9c087e7cf', 'request_id': 'req-1fdd0e87-b688-4e2b-999b-f0a89ef87799'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:15.503672 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] The token cache is being invalidate because project 1cd84d3bad1e444cbeba84c9c087e7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:15.504054 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:15.506609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1fdd0e87-b688-4e2b-999b-f0a89ef87799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:15.509263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1338/2710] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:15 2022] DELETE /identity/v3/projects/1cd84d3bad1e444cbeba84c9c087e7cf => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:15.514037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:15.515055 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:15.515557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:15.553306 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:15.554730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:15.667074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1373/2711] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:15.679322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:15.679807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:15.680235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:15.718796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:15.728052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:15.747812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1339/2712] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:15.757437 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:15.770555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21006ef7-f6b7-4b86-ad33-fa01733ac2cf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:15.775613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:15.776324 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:15.776613 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:15.776958 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:15.777743 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:15.778059 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:15.779955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:15.794136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1374/2713] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:15.800984 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:15.823489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f543fa8-a756-4689-95d0-d22df0092d16 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:15.829990 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:15.831558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:15.831842 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:15.832112 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:15.832831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:15.841031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:15.894752 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1340/2714] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:15 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:15.897524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:15.933250 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe7984-dcc0-4441-bd78-493c80ee5a05 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.032602 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:16.066037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.066932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.067281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.067627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.068412 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.070717 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.108315 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1375/2715] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:15 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:16.117075 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.134375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15138dd-9c14-4f0a-9382-f108bb9edf8c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.140555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.141795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.142154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.142615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.144110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.146296 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.160349 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1341/2716] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:16.164459 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.182002 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182e596-f4c3-44c6-9f63-3dafa22f1880 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.187956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.188939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.189297 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.189645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.191407 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.196561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384ac7d-4848-4ce5-b42c-ce21507c3f81 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.222193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.223629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1376/2717] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:16.235409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca50d0-ff24-4321-b143-5bebd8640f1d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.245608 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.246591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.246953 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.247304 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] PATH_INFO: `/v3/projects/25ba36b0e45e4f7ea45cd9003d1b3163/users/e1a8283601224b2fa16d528e72af6837/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.268795 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ba36b0e45e4f7ea45cd9003d1b3163, user_id=e1a8283601224b2fa16d528e72af6837, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.272116 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.294670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1342/2718] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:16 2022] PUT /identity/v3/projects/25ba36b0e45e4f7ea45cd9003d1b3163/users/e1a8283601224b2fa16d528e72af6837/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:16.307473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.307473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.307959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.376106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] MFA Rules not processed for user `e1a8283601224b2fa16d528e72af6837`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:16.377479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:16.549276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1377/2719] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:16.552196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.708765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:16.756723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f52a9-24ee-41f5-bf45-330d6665459c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.762158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.762900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.763188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.763467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.764195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.766229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.812621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1343/2720] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:16.816788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.835888 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162ccd8-a10d-4db3-85bf-886eadd44cb2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.985025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.986199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.986830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.987454 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.988715 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.992272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.024590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1378/2721] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:16 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:17.028920 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:17.041919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947684-2ba4-409a-a1e5-9a379da1da4d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:17.048199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:17.048901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.049221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.049504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.050306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:17.051997 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.069749 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1344/2722] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:17.071518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:17.084373 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ad4f-9ac0-450c-941a-ff039eb47830 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:17.091294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:17.092346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.092724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.093115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] PATH_INFO: `/v3/projects/91a9c15474f84335ac04a1edf213b7ea/users/bb8485eb81614a75bd9bb8837c228b13/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.118731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a9c15474f84335ac04a1edf213b7ea, user_id=bb8485eb81614a75bd9bb8837c228b13, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:17.124611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.152277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1379/2723] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:17 2022] PUT /identity/v3/projects/91a9c15474f84335ac04a1edf213b7ea/users/bb8485eb81614a75bd9bb8837c228b13/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:17.159388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:17.182337 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ac7a-81cd-41a8-a0c0-f4ac3550ac6b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:17.349301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:17.350710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.351315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.351901 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.354223 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:17.358457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f825-c480-48c4-b315-cf220d380cca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.380409 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1345/2724] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:17.386968 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:17.414814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942577f-4055-4126-9b7c-142b7f4b68f5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:17.436873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:17.438156 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.438535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.438929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] PATH_INFO: `/v3/projects/91a9c15474f84335ac04a1edf213b7ea/users/bb8485eb81614a75bd9bb8837c228b13/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.443335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a9c15474f84335ac04a1edf213b7ea, user_id=bb8485eb81614a75bd9bb8837c228b13, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:17.448215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.457927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:17.486366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1380/2725] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:17 2022] PUT /identity/v3/projects/91a9c15474f84335ac04a1edf213b7ea/users/bb8485eb81614a75bd9bb8837c228b13/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:17.508500 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.509196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.509580 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.581562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] MFA Rules not processed for user `bb8485eb81614a75bd9bb8837c228b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:17.583881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:17.713881 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1346/2726] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:17.726742 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:17.730570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:17.811590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cef3e-108d-434b-9d2d-c1a6464e6d1c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:17.816097 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:17.816920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:17.817302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:17.817854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:17.830733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:17.830733 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1619a0-b4ae-4993-80ff-0ebc4ffd0aa4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:17.853442 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1381/2727] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:18.861459 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:18.927682 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:19.008067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21872c5d-8409-4d26-9770-6f2b48b2e35b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:19.023202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:19.024569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:19.024982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:19.025406 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] PATH_INFO: `/v3/users/e1a8283601224b2fa16d528e72af6837` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:19.028015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a8283601224b2fa16d528e72af6837)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:19.031139 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:19.160446 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a8283601224b2fa16d528e72af6837'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:19.176484 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a8283601224b2fa16d528e72af6837'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:19.207736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a8283601224b2fa16d528e72af6837'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:19.210019 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a8283601224b2fa16d528e72af6837'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:19.210732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a8283601224b2fa16d528e72af6837'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:19.234753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1347/2728] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:18 2022] DELETE /identity/v3/users/e1a8283601224b2fa16d528e72af6837 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:19.241049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:19.242323 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:19.242670 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:19.295963 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:19.297676 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:19.378950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1382/2729] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:19.390790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:19.392370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:19.498360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69cc3-f91c-4124-8a74-c1db1605cdc6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:19.505175 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:19.506134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:19.506966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:19.507307 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:19.531632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:19.534762 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:19.564597 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1348/2730] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:20.088516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:20.090166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:20.119565 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c2eaf-ee89-4e4a-98ed-892fa0946630 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:20.138136 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:20.139185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:20.139480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:20.139758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] PATH_INFO: `/v3/projects/25ba36b0e45e4f7ea45cd9003d1b3163` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:20.141327 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ba36b0e45e4f7ea45cd9003d1b3163)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:20.144221 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:20.246551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ba36b0e45e4f7ea45cd9003d1b3163', 'request_id': 'req-32e3adae-3015-43a0-9f8a-8072540326af'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.256111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ba36b0e45e4f7ea45cd9003d1b3163', 'request_id': 'req-32e3adae-3015-43a0-9f8a-8072540326af'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.274819 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ba36b0e45e4f7ea45cd9003d1b3163', 'request_id': 'req-32e3adae-3015-43a0-9f8a-8072540326af'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.291630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] The token cache is being invalidate because project 25ba36b0e45e4f7ea45cd9003d1b3163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:20.292074 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.292833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.294496 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1383/2731] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:20 2022] DELETE /identity/v3/projects/25ba36b0e45e4f7ea45cd9003d1b3163 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:20.299828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:20.312525 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698135d-1d60-405b-b17b-076d704b5902 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:20.395710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:20.418041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:20.419044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:20.419344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:20.419659 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] PATH_INFO: `/v3/users/bb8485eb81614a75bd9bb8837c228b13` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:20.421351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8485eb81614a75bd9bb8837c228b13)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:20.423402 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:20.504442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8485eb81614a75bd9bb8837c228b13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.516697 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8485eb81614a75bd9bb8837c228b13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.537998 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8485eb81614a75bd9bb8837c228b13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.538830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8485eb81614a75bd9bb8837c228b13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.539515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4674f793-df92-4ed3-8eed-fdd70680ec9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8485eb81614a75bd9bb8837c228b13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.581165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1349/2732] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:20 2022] DELETE /identity/v3/users/bb8485eb81614a75bd9bb8837c228b13 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:21.120397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.122063 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:21.141928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3adae-3015-43a0-9f8a-8072540326af admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.204019 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:21.266982 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.271885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.273366 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.274201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] PATH_INFO: `/v3/projects/91a9c15474f84335ac04a1edf213b7ea` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.277533 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a9c15474f84335ac04a1edf213b7ea)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.285389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.389122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a9c15474f84335ac04a1edf213b7ea', 'request_id': 'req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.402663 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a9c15474f84335ac04a1edf213b7ea', 'request_id': 'req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.417401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a9c15474f84335ac04a1edf213b7ea', 'request_id': 'req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.437443 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] The token cache is being invalidate because project 91a9c15474f84335ac04a1edf213b7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:21.437822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.439082 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bb5f5e10-e351-4566-b41b-3cf792d6bcc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.441959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1384/2733] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:21 2022] DELETE /identity/v3/projects/91a9c15474f84335ac04a1edf213b7ea => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:21.449026 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.449614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.449779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.481911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:21.484152 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:21.558344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1350/2734] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:21.570369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.570369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.570369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.598088 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:21.600420 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:21.614786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1385/2735] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:21.622062 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.642868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9441d-0462-4e90-bda2-dcf119ab1213 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.656188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.657361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.659004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.659515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.660343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:21.660633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.663226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.685423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1351/2736] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:21.692684 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.706214 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931f2d-ffb3-4592-bc8c-693b77fc0e05 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.711626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.712516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.712867 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.713183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.714010 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.716182 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.753660 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1386/2737] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:21 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:21.755202 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.778147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f051400-6eff-4b91-9671-1f764696bd6c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.856730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:21.873416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.874587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.874878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.875161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.875888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.878777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.905656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1352/2738] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:21 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:21.909595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.923561 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5a99-2060-466a-807c-0d839240e288 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.934705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.936726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.937610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.938499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.940852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.942812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.958471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1387/2739] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:21.960786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.977134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dee0-f552-4484-90f1-57466ae345d3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.983588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.984657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.985163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.985633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.987726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.989771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76952-9e36-44ae-931f-22c1dc5776eb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.007736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1353/2740] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:22.009755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:22.029564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8df9de-0747-4135-885d-64b572387f54 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:22.038185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:22.039879 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.040381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.040950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] PATH_INFO: `/v3/projects/51666fd5a77147b3b2cbfa44760aa652/users/f99b4470c88241f99f228ac795cd4d51/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.062120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51666fd5a77147b3b2cbfa44760aa652, user_id=f99b4470c88241f99f228ac795cd4d51, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:22.065423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.089832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1388/2741] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:22 2022] PUT /identity/v3/projects/51666fd5a77147b3b2cbfa44760aa652/users/f99b4470c88241f99f228ac795cd4d51/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:22.094207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.094630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.094861 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.141457 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] MFA Rules not processed for user `f99b4470c88241f99f228ac795cd4d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:22.143103 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:22.277384 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1354/2742] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:22.304050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:22.305959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:22.375159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934d85-47a7-4bf4-b88a-244ecfd82676 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:22.380885 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:22.381688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.381978 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.382259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.390011 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:22.393272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:22.395058 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75679a68-64ce-4fdd-9fde-c80b9f45cf09 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.408452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1389/2743] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:22.431295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:22.500512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7c86-1854-406f-9929-8c0020e38aee None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:22.505488 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:22.506396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.506719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.507000 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] PATH_INFO: `/v3/users/f99b4470c88241f99f228ac795cd4d51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.509588 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99b4470c88241f99f228ac795cd4d51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:22.511620 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.562970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99b4470c88241f99f228ac795cd4d51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.570748 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99b4470c88241f99f228ac795cd4d51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.576156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:22.585578 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99b4470c88241f99f228ac795cd4d51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.586779 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99b4470c88241f99f228ac795cd4d51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.587504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99b4470c88241f99f228ac795cd4d51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.611011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1355/2744] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:22 2022] DELETE /identity/v3/users/f99b4470c88241f99f228ac795cd4d51 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:22.616928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.617275 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.617624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.639229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:22.641378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:22.745761 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1390/2745] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:22.755923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:22.757583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:22.832859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331ac19-16c7-422d-8aa7-1ea84446186d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:22.836791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:22.837464 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.837867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.838248 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.850354 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:22.852010 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf1df-8146-4521-95fb-5614bec29bef service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.863379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1356/2746] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:23.198692 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.201300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:23.220390 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb228-791e-47f6-ac6f-45b149517f15 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.226333 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.227481 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.227927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.228300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] PATH_INFO: `/v3/projects/51666fd5a77147b3b2cbfa44760aa652` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.233255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51666fd5a77147b3b2cbfa44760aa652)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.237445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.296022 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51666fd5a77147b3b2cbfa44760aa652', 'request_id': 'req-2f03f964-67f2-4e5f-86d7-8719325ffd19'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.303269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51666fd5a77147b3b2cbfa44760aa652', 'request_id': 'req-2f03f964-67f2-4e5f-86d7-8719325ffd19'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.313685 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51666fd5a77147b3b2cbfa44760aa652', 'request_id': 'req-2f03f964-67f2-4e5f-86d7-8719325ffd19'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.327294 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] The token cache is being invalidate because project 51666fd5a77147b3b2cbfa44760aa652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:23.327591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.328318 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2f03f964-67f2-4e5f-86d7-8719325ffd19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.330045 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1391/2747] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:23 2022] DELETE /identity/v3/projects/51666fd5a77147b3b2cbfa44760aa652 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:23.338671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.339214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.339586 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.359461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:23.360718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:23.446084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1357/2748] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:23.456438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d84a7-5f59-407b-808f-859587f33bdd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.456729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d84a7-5f59-407b-808f-859587f33bdd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.457015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d84a7-5f59-407b-808f-859587f33bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.478911 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4d7d84a7-5f59-407b-808f-859587f33bdd None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:23.480846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d7d84a7-5f59-407b-808f-859587f33bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:23.493180 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1392/2749] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:23.498226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.512306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967482bc-fcbd-4c38-9f07-d7eff5304d4c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.513783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.514150 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.514542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.518109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.519070 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.519460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.519855 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.520753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:23.521177 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.523388 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.538174 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1358/2750] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:48:23.541244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] MFA Rules not processed for user `b54a5070a23e45a3975c0fa0f2bcc76e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:23.542552 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:23.545267 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.557957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83eae5-14f8-4fee-a613-f4e525afe8fc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.563780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.564594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.564945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.565227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.566569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.568807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.604479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1359/2751] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:23 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:23.605817 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.628195 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1393/2752] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:23.675738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:23.693154 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b699e77-a05e-48fe-a874-a64276befc85 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.702481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.703834 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.704193 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.705076 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.706235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.710133 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.740568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1360/2753] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:23 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:23.746157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.758518 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21056-335d-4a43-a23c-24e32a962a73 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.763571 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.764578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.765159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.765543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.766509 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.768457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.781053 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1394/2754] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:23.785667 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.803760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6e17c-0414-4f1a-8be8-d3d22b27268f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.813279 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.814675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.815264 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.815833 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.817287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.820927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a66aa-7235-47e9-8d0b-7733bcd9c97b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.834969 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1361/2755] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:23.840184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.853947 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081edd88-3d0f-48bf-b8af-78195eecd13e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.862987 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123049c0-01c2-415b-94da-979df257306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.864756 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123049c0-01c2-415b-94da-979df257306b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.865388 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123049c0-01c2-415b-94da-979df257306b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.865977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123049c0-01c2-415b-94da-979df257306b admin admin] PATH_INFO: `/v3/projects/f614b9264ed14eab9e67ee4fca9b922c/users/34a4e3980f4d4cc98a4be829b351315c/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.870224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:23.899341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123049c0-01c2-415b-94da-979df257306b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f614b9264ed14eab9e67ee4fca9b922c, user_id=34a4e3980f4d4cc98a4be829b351315c, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.904972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123049c0-01c2-415b-94da-979df257306b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.928674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1395/2756] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:23 2022] PUT /identity/v3/projects/f614b9264ed14eab9e67ee4fca9b922c/users/34a4e3980f4d4cc98a4be829b351315c/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:24.015477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73d04-8af3-43ef-9b5c-7114ae424a8c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.016812 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73d04-8af3-43ef-9b5c-7114ae424a8c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.017241 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73d04-8af3-43ef-9b5c-7114ae424a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.070671 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eab73d04-8af3-43ef-9b5c-7114ae424a8c None None] MFA Rules not processed for user `34a4e3980f4d4cc98a4be829b351315c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.072923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eab73d04-8af3-43ef-9b5c-7114ae424a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.169009 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1362/2757] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:24.183782 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123049c0-01c2-415b-94da-979df257306b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.186172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-123049c0-01c2-415b-94da-979df257306b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.251940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123049c0-01c2-415b-94da-979df257306b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.256618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.257308 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.257600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.257884 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.268302 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.269858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d261c-fc9a-4d60-b95d-1069537d70e8 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.284590 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1396/2758] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:25.813237 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.813770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.813770 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.833708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] MFA Rules not processed for user `34a4e3980f4d4cc98a4be829b351315c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:25.835334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.844250 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1363/2759] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:26.101143 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.102564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.103025 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.139723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:26.142199 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.258828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1397/2760] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:26.283992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.286910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.426144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc54af9-3b1e-42ae-bf75-848261c2fde7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.431839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.432521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.432806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.433084 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.443681 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.445701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310106-0a4d-41c5-8f28-e76442817a4f service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.458336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1364/2761] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:48:36.243900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:36.452363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:42.681130 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:42.683118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:42.701212 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a5cc-8e08-4453-9704-f3fed2119689 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:42.705838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:42.706900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:42.707387 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:42.707805 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:42.710367 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:42.797516 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:42.801158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff554f64-ad30-47bd-8ae5-f5013c7cac25 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:42.816851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1398/2762] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:51.036207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:51.037115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:51.037115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:51.057750 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] MFA Rules not processed for user `6c7e0694c5ab46a4adef7a1f7b1fed4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:51.059013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:51.067414 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1365/2763] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:48:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:52.811498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=28 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:55.726052 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47cf5-d05f-4fa5-93f9-c6cdc6d56f9b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.728955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47cf5-d05f-4fa5-93f9-c6cdc6d56f9b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.729618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47cf5-d05f-4fa5-93f9-c6cdc6d56f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.799758 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bfb47cf5-d05f-4fa5-93f9-c6cdc6d56f9b None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:55.802674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bfb47cf5-d05f-4fa5-93f9-c6cdc6d56f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:55.826077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1399/2764] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:48:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:55.846821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.878714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb5617-1047-49ed-8799-7abfaca48ba6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.888554 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.889676 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.890182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.890730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.909626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.912003 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.927392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1366/2765] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:48:56.698638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615be-ac1c-4ffb-a816-7ff44c72a1d9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.698928 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615be-ac1c-4ffb-a816-7ff44c72a1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.699154 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a615be-ac1c-4ffb-a816-7ff44c72a1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.729598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90a615be-ac1c-4ffb-a816-7ff44c72a1d9 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:56.731799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90a615be-ac1c-4ffb-a816-7ff44c72a1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:56.745380 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1400/2766] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:48:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:56.761775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:56.774364 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84367a54-1b2d-4670-9042-690dde79059f service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:56.780042 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:56.780801 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.781155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.781505 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.792526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:56.794494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:56.806355 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1367/2767] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:48:57.097849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ed20-8fac-47dc-8ca4-11c96b984f1c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:57.099039 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ed20-8fac-47dc-8ca4-11c96b984f1c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:57.100239 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3ed20-8fac-47dc-8ca4-11c96b984f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:57.143377 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-60c3ed20-8fac-47dc-8ca4-11c96b984f1c None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:57.146880 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-60c3ed20-8fac-47dc-8ca4-11c96b984f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:57.166849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1401/2768] 10.222.0.77 () {58 vars in 1009 bytes} [Mon Jul 18 16:48:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:57.183246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:57.196460 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8f674-4b72-4290-9cfb-fcf817f7317c service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:57.202717 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:57.203408 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:57.203691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:57.204242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:57.214930 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:57.216623 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:57.227110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1368/2769] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:48:57.688455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:57.688800 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:57.689125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:57.708339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:57.709709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:57.717851 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1402/2770] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:48:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:57.731591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:57.744087 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad41b1-98a6-40ff-aee8-1fb1be837190 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:57.750124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:57.750854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:57.751143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:57.751425 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:57.761792 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:57.763403 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:57.773973 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1369/2771] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:48:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:49:00.311346 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:00.412598 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36282b1-51e3-4c3b-b6c6-5633546e228d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:00.419185 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:00.420921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.421395 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.421860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] PATH_INFO: `/v3/users/34a4e3980f4d4cc98a4be829b351315c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.424612 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a4e3980f4d4cc98a4be829b351315c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:00.428173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:00.501012 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a4e3980f4d4cc98a4be829b351315c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:00.511495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a4e3980f4d4cc98a4be829b351315c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:00.528555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a4e3980f4d4cc98a4be829b351315c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:00.529033 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:00.531109 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.539868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a4e3980f4d4cc98a4be829b351315c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:00.540864 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4815eeee-3c9c-4292-90ef-ee1cefedc8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a4e3980f4d4cc98a4be829b351315c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:00.549889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1403/2772] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:00 2022] DELETE /identity/v3/users/34a4e3980f4d4cc98a4be829b351315c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:00.555299 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.555788 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.556168 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.578452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:00.579812 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.621366 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde56a9-d446-4e31-927b-85d50a02201f service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:00.639077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:00.671619 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1404/2773] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:00.681162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:00.683048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.704856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:00.705653 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.706105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.706374 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.713065 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.773882 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c350c90-9ad7-4afb-983a-6e4151f6e620 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:00.780410 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:00.781810 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.782280 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.782778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.807282 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:00.810111 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:00.819069 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:00.821524 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5b7c-ba73-4712-8d42-95fac0c6cb41 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:00.827823 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1405/2774] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:49:00.839158 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1370/2775] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:01.136272 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:01.137975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:01.167339 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a622884-bf58-4a31-a77c-cba2019def56 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:01.196883 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:01.198008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:01.199135 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:01.200172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] PATH_INFO: `/v3/projects/f614b9264ed14eab9e67ee4fca9b922c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:01.203139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f614b9264ed14eab9e67ee4fca9b922c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:01.205423 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:01.339417 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f614b9264ed14eab9e67ee4fca9b922c', 'request_id': 'req-64f8e739-8c8f-483c-a8da-fa22a63d8edf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.352764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f614b9264ed14eab9e67ee4fca9b922c', 'request_id': 'req-64f8e739-8c8f-483c-a8da-fa22a63d8edf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.368825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f614b9264ed14eab9e67ee4fca9b922c', 'request_id': 'req-64f8e739-8c8f-483c-a8da-fa22a63d8edf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.404381 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] The token cache is being invalidate because project f614b9264ed14eab9e67ee4fca9b922c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:01.405737 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.409244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.417853 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1406/2776] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:01 2022] DELETE /identity/v3/projects/f614b9264ed14eab9e67ee4fca9b922c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:10.831563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:11.392645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:11.546995 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:11.547392 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:11.547780 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:11.567269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] MFA Rules not processed for user `d66dde93e0f14ce799c9c6294f216781`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:11.568535 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:11.649417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1371/2777] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:49:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:13.535277 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:13.536776 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:13.655087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8e739-8c8f-483c-a8da-fa22a63d8edf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:13.662634 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:13.664721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:13.665469 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:13.666157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:13.684260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:13.797908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:13.800972 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:13.821204 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1407/2778] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:21.638176 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=25 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:23.811122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=50 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:30.565611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:30.577583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548ae42-40a7-4555-8ee4-eb2a4fec88d3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:30.582712 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:30.583509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:30.583934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:30.584325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:30.586626 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:30.680089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:30.681665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:30.695587 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1372/2779] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:31.940977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:31.942956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:31.955638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d061c7-db79-44c2-8c2b-6ac42f85b6bc service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:31.959587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:31.960240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:31.960524 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:31.960797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:31.963131 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:31.979359 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:31.980811 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:31.993917 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1408/2780] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:40.690471 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=34 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:41.988287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:04.703328 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.715481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ea1e-d4a3-45e3-a148-37b8f2581a82 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.719563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.720277 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.720562 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.720844 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.722927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.733992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.736157 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.745735 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1373/2781] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:14.741475 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:22.262528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:22.266210 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:22.309950 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226cbaa-135e-4c86-8a8f-64aa275e368d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:22.334471 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:22.335706 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:22.336215 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:22.336688 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:22.339897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:22.357181 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:22.359066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27259-9ae5-4f90-8db7-5193355956d0 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:22.379940 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1409/2782] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:27.298991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:27.300670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:27.317961 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4c55b-9782-4aa3-93d3-7aa755d66268 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:27.323791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:27.324463 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:27.324746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:27.325023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:27.327023 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:27.339489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:27.341156 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:27.352300 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1374/2783] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:32.368872 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:32.977227 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.977754 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.978458 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.072606 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] MFA Rules not processed for user `e98cf2fa9bee4dcfa9f7410d19b67f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:33.076547 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:33.162629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1410/2784] 10.222.0.77 () {58 vars in 967 bytes} [Mon Jul 18 16:50:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:33.235883 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.237429 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:33.356143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de7add-5154-4784-b5f1-090c1eb0c9ae service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.361814 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71aecafdbfad40d4979f6f6358a58109', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.362531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.362816 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.363091 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.372956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.374665 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.384594 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1375/2785] 10.222.0.77 () {60 vars in 1384 bytes} [Mon Jul 18 16:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:50:37.435832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:37.455929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da6949-752d-455b-a648-b3ac236d6902 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:37.464707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:37.465638 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:37.466138 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:37.466603 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:37.468873 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:37.544996 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:37.546745 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:37.558651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1411/2786] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:42.389377 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.394784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:42.519685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548d34-2805-49a4-b2c5-75f7fdd32ecc service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.519685 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.520468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.520971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:42.529632 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.529965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.530182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.535615 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:42.539587 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43aa4c7-40f3-401c-b436-f96639a64632 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.544740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.545910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.546517 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.547133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] PATH_INFO: `/v3/users/b54a5070a23e45a3975c0fa0f2bcc76e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.549241 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54a5070a23e45a3975c0fa0f2bcc76e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.551702 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.570040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.573057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.605389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1376/2787] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:42.654624 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54a5070a23e45a3975c0fa0f2bcc76e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:42.674201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54a5070a23e45a3975c0fa0f2bcc76e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:42.680578 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:42.698625 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54a5070a23e45a3975c0fa0f2bcc76e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:42.712528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54a5070a23e45a3975c0fa0f2bcc76e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:42.713644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54a5070a23e45a3975c0fa0f2bcc76e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:42.723508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1412/2788] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:42 2022] DELETE /identity/v3/users/b54a5070a23e45a3975c0fa0f2bcc76e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:42.965946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.989315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a617-854f-4fba-9d15-601580143284 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.061768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:43.090890 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.093489 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.095656 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.096155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] PATH_INFO: `/v3/projects/6343e3024ef0469da293401317da07f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.098768 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6343e3024ef0469da293401317da07f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.102339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.126032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.128697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:43.154118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975351-0790-4d0a-9d5f-c0b3342b7460 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.170907 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6343e3024ef0469da293401317da07f2', 'request_id': 'req-e079d79e-7507-480a-8089-1be5c1414ff0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.184260 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6343e3024ef0469da293401317da07f2', 'request_id': 'req-e079d79e-7507-480a-8089-1be5c1414ff0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.194297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6343e3024ef0469da293401317da07f2', 'request_id': 'req-e079d79e-7507-480a-8089-1be5c1414ff0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.207284 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] The token cache is being invalidate because project 6343e3024ef0469da293401317da07f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:43.207644 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.208324 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.209989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1377/2789] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:42 2022] DELETE /identity/v3/projects/6343e3024ef0469da293401317da07f2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:43.217344 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.218088 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:43.249189 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.251446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.251446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.251446 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] PATH_INFO: `/v3/users/d66dde93e0f14ce799c9c6294f216781` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.253147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66dde93e0f14ce799c9c6294f216781)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.255184 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.322886 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66dde93e0f14ce799c9c6294f216781'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.332393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66dde93e0f14ce799c9c6294f216781'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.333278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079d79e-7507-480a-8089-1be5c1414ff0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.340379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.341378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.341799 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.342270 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] PATH_INFO: `/v3/users/fcde37d1d66745f0938f158eb32e10c9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.344120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcde37d1d66745f0938f158eb32e10c9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.346744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.347728 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66dde93e0f14ce799c9c6294f216781'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.357808 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66dde93e0f14ce799c9c6294f216781'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.358616 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66dde93e0f14ce799c9c6294f216781'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.365976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1413/2790] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:43 2022] DELETE /identity/v3/users/d66dde93e0f14ce799c9c6294f216781 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:43.413063 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcde37d1d66745f0938f158eb32e10c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.421221 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcde37d1d66745f0938f158eb32e10c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.435047 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcde37d1d66745f0938f158eb32e10c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.436167 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcde37d1d66745f0938f158eb32e10c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.437226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcde37d1d66745f0938f158eb32e10c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.444407 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:43.458243 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1378/2791] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:43 2022] DELETE /identity/v3/users/fcde37d1d66745f0938f158eb32e10c9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:43.635652 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.637162 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:43.712853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.715571 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:43.720971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:43.725690 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83327d24-429a-4bae-97bb-0663b7bdcda1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.805811 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d5019-21fd-40dd-b9ce-2a216b35152b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.812040 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.813504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.813669 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.813796 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.814053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] PATH_INFO: `/v3/projects/1b1a074500614af99c90dee2ffb7dfbc` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.814444 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.814741 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.815021 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] PATH_INFO: `/v3/projects/628c00a1a5df481982b083fa3c447be2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.815320 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1a074500614af99c90dee2ffb7dfbc)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.816276 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=628c00a1a5df481982b083fa3c447be2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.817263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.818190 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.867831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1a074500614af99c90dee2ffb7dfbc', 'request_id': 'req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.874902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628c00a1a5df481982b083fa3c447be2', 'request_id': 'req-9ca47822-2c53-4029-9375-64dd64aefa0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.878619 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1a074500614af99c90dee2ffb7dfbc', 'request_id': 'req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.882562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628c00a1a5df481982b083fa3c447be2', 'request_id': 'req-9ca47822-2c53-4029-9375-64dd64aefa0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.885646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1a074500614af99c90dee2ffb7dfbc', 'request_id': 'req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.890956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628c00a1a5df481982b083fa3c447be2', 'request_id': 'req-9ca47822-2c53-4029-9375-64dd64aefa0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.899197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] The token cache is being invalidate because project 1b1a074500614af99c90dee2ffb7dfbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:43.899487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.900137 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e8b1048a-c7c5-4b3d-aed5-73dff7180baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.901678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1379/2792] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:43 2022] DELETE /identity/v3/projects/1b1a074500614af99c90dee2ffb7dfbc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:43.903539 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] The token cache is being invalidate because project 628c00a1a5df481982b083fa3c447be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:43.903837 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.904504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9ca47822-2c53-4029-9375-64dd64aefa0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:43.906099 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1414/2793] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:43 2022] DELETE /identity/v3/projects/628c00a1a5df481982b083fa3c447be2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:43.911747 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.912128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.912515 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.943919 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:43.945565 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:44.040888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1380/2794] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:44.050896 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.051305 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.051588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.072001 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:44.073268 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:44.082065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1415/2795] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:44.095708 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.107569 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dacf65-7a76-445b-bc55-e5c8faf59b03 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.113480 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.114165 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.114486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.114786 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.115700 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:44.116008 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.117732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.135060 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1381/2796] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:44.139768 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.152617 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b7397-8e75-47ba-8411-01bf957e1bb7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.158522 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.159403 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.159763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.160115 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.160915 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.163094 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.193244 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1416/2797] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:44.197527 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.208149 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f720-cf09-4985-b465-03088068c42c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.217789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:44.273980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.274764 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.275108 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.275423 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.276181 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.278467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.302044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1382/2798] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:44.306945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.318398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a2b12-be87-480a-bbad-1c97b3cdc15b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.323643 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.324336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.324627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.325014 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.325781 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.327452 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.338954 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1417/2799] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:44.344461 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.355994 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f15154-0ad8-48c3-a7a5-7569fb4783e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.361528 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.362400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.362715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.362996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.363742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.365389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a38fc4-5816-4c93-9732-8800ce882f07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.376991 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1383/2800] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:44.382031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.393774 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb0777-f52a-4ddf-a194-7365bc5f4033 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.399429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.400370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.400656 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.400936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] PATH_INFO: `/v3/projects/1580c2a59e904eae9cca26b0ff17ac6c/users/8d86cd1dec2847d996502eafb1486f4f/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.419438 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1580c2a59e904eae9cca26b0ff17ac6c, user_id=8d86cd1dec2847d996502eafb1486f4f, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.422630 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.438729 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1418/2801] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:44 2022] PUT /identity/v3/projects/1580c2a59e904eae9cca26b0ff17ac6c/users/8d86cd1dec2847d996502eafb1486f4f/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:44.449115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.449956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.450126 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.481031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] MFA Rules not processed for user `8d86cd1dec2847d996502eafb1486f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:44.482363 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:44.576323 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1384/2802] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:44.582038 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.623495 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:44.663687 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2938-73d9-4f73-81a9-8102cf71a88e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.668697 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.669378 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.669703 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.669986 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.671160 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.673226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.702057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1419/2803] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:44.708487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.720879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c247aea-667e-4900-9096-1c46deb92e13 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.788604 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.789294 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.789579 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.789859 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.790610 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.792618 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.815161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1385/2804] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:44 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:44.820166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.832530 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802117aa-9102-4343-831e-742959c9eb03 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.838707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.839468 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.839798 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.840166 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.841027 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.842858 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.854856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1420/2805] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:44.859310 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.870741 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c84eca-9fa3-43d3-a21c-3dbbf7b3ba23 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:44.877417 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:44.878509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:44.878878 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:44.879217 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] PATH_INFO: `/v3/projects/16e6eff24cda44528aea4a7477f95ed3/users/89eff2ebaa454abbb99ae1ea00aa6e44/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:44.889479 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:44.898511 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e6eff24cda44528aea4a7477f95ed3, user_id=89eff2ebaa454abbb99ae1ea00aa6e44, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:44.901393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:44.916982 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1386/2806] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:44 2022] PUT /identity/v3/projects/16e6eff24cda44528aea4a7477f95ed3/users/89eff2ebaa454abbb99ae1ea00aa6e44/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:44.922720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:44.933582 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d7de9-b12b-48f2-9e0c-959b6156c8f1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:45.013831 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:45.014572 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.014874 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.015174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.016528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:45.018159 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd74789-a851-4860-91fa-7494c4b1cb88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:45.029158 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1421/2807] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:45.033791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:45.044758 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eca29e-a3a3-4362-b067-4926fc5a1658 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:45.050131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:45.051097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.051375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.051647 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] PATH_INFO: `/v3/projects/16e6eff24cda44528aea4a7477f95ed3/users/89eff2ebaa454abbb99ae1ea00aa6e44/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.053911 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e6eff24cda44528aea4a7477f95ed3, user_id=89eff2ebaa454abbb99ae1ea00aa6e44, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:45.056868 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:45.069182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1387/2808] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:45 2022] PUT /identity/v3/projects/16e6eff24cda44528aea4a7477f95ed3/users/89eff2ebaa454abbb99ae1ea00aa6e44/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:45.080364 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.080658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.080936 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.109977 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] MFA Rules not processed for user `89eff2ebaa454abbb99ae1ea00aa6e44`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:45.111281 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:45.192508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1422/2809] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:45.203879 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:45.205624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:45.271197 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48079b0-e57b-427d-b727-c7dd8f5a797e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:45.275521 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:45.276215 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.276443 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.276723 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.286782 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:45.288244 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe5fc4-fb65-4422-8c0e-129dfa03fbc3 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:45.300107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1388/2810] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:45.449633 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:45.489521 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:45.525847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf0a1a-e7a0-482d-baa5-9ed7fc17e6f1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:45.530592 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:45.531976 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.532256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.532528 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] PATH_INFO: `/v3/users/8d86cd1dec2847d996502eafb1486f4f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.533828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d86cd1dec2847d996502eafb1486f4f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:45.535802 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:45.589929 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d86cd1dec2847d996502eafb1486f4f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:45.600120 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d86cd1dec2847d996502eafb1486f4f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:45.614376 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d86cd1dec2847d996502eafb1486f4f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:45.624532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d86cd1dec2847d996502eafb1486f4f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:45.625636 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d86cd1dec2847d996502eafb1486f4f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:45.633062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1423/2811] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:45 2022] DELETE /identity/v3/users/8d86cd1dec2847d996502eafb1486f4f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:45.638784 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.639442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.639442 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.659721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:45.661015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:45.727989 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1389/2812] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:45.738908 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:45.740629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:45.803455 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be800f31-d0c8-499c-b4a6-876ecfb52da3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:45.807314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:45.807981 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:45.808263 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:45.808540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:45.819061 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:45.820627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:45.831436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1424/2813] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:46.179943 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:46.181611 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:46.201145 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0628f-c28e-42ae-a3f3-447d02f015d5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:46.209192 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:46.210763 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:46.211272 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:46.211561 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] PATH_INFO: `/v3/projects/1580c2a59e904eae9cca26b0ff17ac6c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:46.213072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1580c2a59e904eae9cca26b0ff17ac6c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:46.215469 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:46.273011 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1580c2a59e904eae9cca26b0ff17ac6c', 'request_id': 'req-e6e2f46c-aa7f-4567-b308-c930a495f242'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.283052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1580c2a59e904eae9cca26b0ff17ac6c', 'request_id': 'req-e6e2f46c-aa7f-4567-b308-c930a495f242'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.291224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1580c2a59e904eae9cca26b0ff17ac6c', 'request_id': 'req-e6e2f46c-aa7f-4567-b308-c930a495f242'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.300393 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:46.305201 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] The token cache is being invalidate because project 1580c2a59e904eae9cca26b0ff17ac6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:46.305573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.307115 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.308996 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1390/2814] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:46 2022] DELETE /identity/v3/projects/1580c2a59e904eae9cca26b0ff17ac6c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:46.314436 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:46.325004 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef8496-8a6b-4e57-b33c-a0c1d614eff8 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:46.378172 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:46.392849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:46.393797 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:46.394084 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:46.394363 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] PATH_INFO: `/v3/users/89eff2ebaa454abbb99ae1ea00aa6e44` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:46.395698 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89eff2ebaa454abbb99ae1ea00aa6e44)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:46.397666 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:46.452477 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89eff2ebaa454abbb99ae1ea00aa6e44'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.460891 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89eff2ebaa454abbb99ae1ea00aa6e44'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.474144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89eff2ebaa454abbb99ae1ea00aa6e44'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.483860 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89eff2ebaa454abbb99ae1ea00aa6e44'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.484739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af992773-20bf-4f57-9403-80a175ddb1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89eff2ebaa454abbb99ae1ea00aa6e44'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.492116 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1425/2815] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:46 2022] DELETE /identity/v3/users/89eff2ebaa454abbb99ae1ea00aa6e44 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:46.814098 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:46.817234 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:46.835332 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f46c-aa7f-4567-b308-c930a495f242 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:46.911187 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:46.911976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:46.912266 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:46.912539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] PATH_INFO: `/v3/projects/16e6eff24cda44528aea4a7477f95ed3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:46.913794 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e6eff24cda44528aea4a7477f95ed3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:46.915807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:46.974341 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e6eff24cda44528aea4a7477f95ed3', 'request_id': 'req-9647d29a-1ecc-4758-9480-9fa8e7b82944'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.983325 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e6eff24cda44528aea4a7477f95ed3', 'request_id': 'req-9647d29a-1ecc-4758-9480-9fa8e7b82944'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:46.991985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e6eff24cda44528aea4a7477f95ed3', 'request_id': 'req-9647d29a-1ecc-4758-9480-9fa8e7b82944'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:47.012148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] The token cache is being invalidate because project 16e6eff24cda44528aea4a7477f95ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:47.012612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:47.013687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9647d29a-1ecc-4758-9480-9fa8e7b82944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:47.016274 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1391/2816] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:46 2022] DELETE /identity/v3/projects/16e6eff24cda44528aea4a7477f95ed3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:47.024078 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.024645 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.025051 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.052034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:47.053440 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:47.124101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1426/2817] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:47.134508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.134876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.135297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.163458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:47.165327 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:47.180130 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1392/2818] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:47.185542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.196295 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf8bf-5a2b-4922-9a52-f3f0b16c73b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.201379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.202075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.202427 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.202712 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.203478 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:47.203789 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.205479 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.219909 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1427/2819] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:47.224813 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.238657 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299adcb-5587-44ac-a980-8e7a3b942f71 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.246609 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.247715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.248184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.248663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.249949 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.253548 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.293630 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1393/2820] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:47.300360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.310921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee6c6-0933-4d65-92c4-3729a3317d8d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.368983 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:47.379197 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.380256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.381008 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.381562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.382922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.385629 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.411246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1428/2821] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:47.416207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.428526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48ca7-deff-48dc-95c0-e55b4d0c8ce5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.433806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.434658 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.434988 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.435315 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.436105 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.438222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.451510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1394/2822] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:47.456512 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.467707 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798df647-6ba2-4ea5-aa76-67f77fc33049 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.473336 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.474161 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.474537 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.474871 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.475749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.477489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeaa17-dc9c-4de3-9953-15659660ca3a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.489191 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1429/2823] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:47.494007 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.505428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4032-f99c-481f-9577-6cb62b025a80 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.510716 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.511675 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.511993 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.512306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] PATH_INFO: `/v3/projects/c82eb09668b04a879c639e25a4e48199/users/d48d69d100a94ef8a5dea89ea66d4d79/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.530699 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82eb09668b04a879c639e25a4e48199, user_id=d48d69d100a94ef8a5dea89ea66d4d79, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.534044 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.549061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1395/2824] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:47 2022] PUT /identity/v3/projects/c82eb09668b04a879c639e25a4e48199/users/d48d69d100a94ef8a5dea89ea66d4d79/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:47.559840 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.560127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.560397 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.604193 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] MFA Rules not processed for user `d48d69d100a94ef8a5dea89ea66d4d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:47.605543 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:47.684507 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1430/2825] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:47.689709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.729504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:47.770851 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44af7-c12a-428a-832a-0866eb16f859 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.776537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.777224 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.777510 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.777783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.778551 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.780695 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.811777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1396/2826] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:47.816825 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.829832 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6701b4-a3bd-4e32-b4fe-5960aaedba26 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.902428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.903278 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.903828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.904260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.905037 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.907925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.933203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1431/2827] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:47 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:47.938731 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.950301 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d9d9-826f-4c17-947a-0a56dbaff847 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.955850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.956557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.956850 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.957124 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:47.957873 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:47.959555 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:47.972055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1397/2828] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:47.976402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:47.989551 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172ee48-8e4d-4348-8895-63507c3c8f93 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:47.994898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:47.995830 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:47.996125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:47.996402 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] PATH_INFO: `/v3/projects/ae4618c507a7449c9b13ecf344aff5d4/users/8f931d5481f94b6dbed8adc9f727f6c3/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.014875 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4618c507a7449c9b13ecf344aff5d4, user_id=8f931d5481f94b6dbed8adc9f727f6c3, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:48.017586 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:48.033332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1432/2829] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:47 2022] PUT /identity/v3/projects/ae4618c507a7449c9b13ecf344aff5d4/users/8f931d5481f94b6dbed8adc9f727f6c3/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:48.037857 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:48.050790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899282e2-5d53-43a7-adb5-bd8b6c213085 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:48.124977 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:48.125802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:48.126161 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:48.126477 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.127226 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:48.128854 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c4cf1-59ca-4341-bed2-746ff7f07f2e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:48.141734 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1398/2830] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:48.146262 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:48.158256 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c74c-f119-41a3-baee-f5282f7f6eb1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:48.163544 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:48.164489 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:48.164894 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:48.165183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] PATH_INFO: `/v3/projects/ae4618c507a7449c9b13ecf344aff5d4/users/8f931d5481f94b6dbed8adc9f727f6c3/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.167456 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4618c507a7449c9b13ecf344aff5d4, user_id=8f931d5481f94b6dbed8adc9f727f6c3, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:48.170822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:48.182975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1433/2831] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:48 2022] PUT /identity/v3/projects/ae4618c507a7449c9b13ecf344aff5d4/users/8f931d5481f94b6dbed8adc9f727f6c3/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:48.195147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:48.195373 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:48.195624 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.226625 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] MFA Rules not processed for user `8f931d5481f94b6dbed8adc9f727f6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:48.227875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:48.310333 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1399/2832] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:48.323780 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:48.326659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:48.362614 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:48.408044 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efdb52-239a-48a8-b0a9-efdc88f6f2b7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:48.412639 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:48.413296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:48.413594 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:48.413856 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.424777 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:48.426248 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ff56e-23e5-4ba1-a303-3ae57ef4199e service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:48.437147 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1434/2833] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:48.843771 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:48.874351 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:48.944144 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd7fd7-a69f-4a38-a59c-41ef36b73905 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:48.949870 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:48.950874 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:48.951189 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:48.951506 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] PATH_INFO: `/v3/users/d48d69d100a94ef8a5dea89ea66d4d79` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:48.952945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48d69d100a94ef8a5dea89ea66d4d79)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:48.955134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:49.012110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48d69d100a94ef8a5dea89ea66d4d79'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.019493 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48d69d100a94ef8a5dea89ea66d4d79'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.033933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48d69d100a94ef8a5dea89ea66d4d79'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.034736 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48d69d100a94ef8a5dea89ea66d4d79'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.035356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48d69d100a94ef8a5dea89ea66d4d79'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.053622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1400/2834] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:48 2022] DELETE /identity/v3/users/d48d69d100a94ef8a5dea89ea66d4d79 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:49.059087 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:49.059967 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:49.060137 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:49.080429 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:49.081649 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:49.146962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1435/2835] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:49.158278 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:49.159853 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:49.222808 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73746c69-8216-4054-ba85-5a8ded4cd115 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:49.226591 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:49.227252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:49.227531 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:49.227809 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:49.238306 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:49.239793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:49.251185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1401/2836] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:49.596473 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:49.599096 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:49.619269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de1aff-6f38-463c-9fec-5f1a639ec0ba None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:49.625352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:49.626627 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:49.626980 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:49.627351 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] PATH_INFO: `/v3/projects/c82eb09668b04a879c639e25a4e48199` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:49.628843 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82eb09668b04a879c639e25a4e48199)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:49.632035 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:49.694480 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82eb09668b04a879c639e25a4e48199', 'request_id': 'req-ca5a104e-024f-4ef3-9fbc-9a9277116341'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.701759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82eb09668b04a879c639e25a4e48199', 'request_id': 'req-ca5a104e-024f-4ef3-9fbc-9a9277116341'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.712201 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82eb09668b04a879c639e25a4e48199', 'request_id': 'req-ca5a104e-024f-4ef3-9fbc-9a9277116341'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.725900 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] The token cache is being invalidate because project c82eb09668b04a879c639e25a4e48199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:49.726192 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.726893 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.728556 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1436/2837] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:49 2022] DELETE /identity/v3/projects/c82eb09668b04a879c639e25a4e48199 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:49.733169 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:49.744086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82f9e-d716-4202-927d-3d041dc7d97c service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:49.795219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:49.814242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:49.815184 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:49.815462 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:49.815738 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] PATH_INFO: `/v3/users/8f931d5481f94b6dbed8adc9f727f6c3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:49.817094 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f931d5481f94b6dbed8adc9f727f6c3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:49.819138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:49.870122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f931d5481f94b6dbed8adc9f727f6c3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.878430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f931d5481f94b6dbed8adc9f727f6c3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.893508 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f931d5481f94b6dbed8adc9f727f6c3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.894517 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f931d5481f94b6dbed8adc9f727f6c3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.895190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38864ef2-72ba-4004-9ce0-471c81fee1aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f931d5481f94b6dbed8adc9f727f6c3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:49.912097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1402/2838] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:49 2022] DELETE /identity/v3/users/8f931d5481f94b6dbed8adc9f727f6c3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:50.245827 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.247727 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:50.275719 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a104e-024f-4ef3-9fbc-9a9277116341 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.348303 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:50.371439 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.372644 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.373018 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.373396 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] PATH_INFO: `/v3/projects/ae4618c507a7449c9b13ecf344aff5d4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.374931 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4618c507a7449c9b13ecf344aff5d4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.377535 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.430383 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4618c507a7449c9b13ecf344aff5d4', 'request_id': 'req-cac1bd18-4423-49d8-8567-039ac256a18d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.438699 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4618c507a7449c9b13ecf344aff5d4', 'request_id': 'req-cac1bd18-4423-49d8-8567-039ac256a18d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.447749 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4618c507a7449c9b13ecf344aff5d4', 'request_id': 'req-cac1bd18-4423-49d8-8567-039ac256a18d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.458813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] The token cache is being invalidate because project ae4618c507a7449c9b13ecf344aff5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:50.459105 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.459763 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cac1bd18-4423-49d8-8567-039ac256a18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.461343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1437/2839] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:50 2022] DELETE /identity/v3/projects/ae4618c507a7449c9b13ecf344aff5d4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:50.467356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.467876 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.468389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.496106 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:50.497359 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:50.566339 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1403/2840] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:50.574269 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.574583 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.574962 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.599486 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:50.601030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:50.610854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1438/2841] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:50.618041 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.630678 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfc1e2-97c0-420b-8993-c07ed689165d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.636247 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.636933 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.637210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.637485 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.638220 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:50.638557 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.640238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.653761 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1404/2842] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:50.659984 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.670352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ad7b9-abad-44f4-987f-bc16192c34a2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.676597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.677291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.677580 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.677852 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.678605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.680596 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.718331 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1439/2843] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:50.725014 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.736830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc6a5e-7042-4d9c-a5a1-99ca3b5889e8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.789928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:50.803987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.804909 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.805194 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.805470 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.806178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.808188 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.829365 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1405/2844] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:50.833907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.849029 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd652a9-187e-4e03-bd82-1e49bed437c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.854600 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.855259 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.855542 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.855822 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.856597 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.858369 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.871919 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1440/2845] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:50.876246 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.887871 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091e365-a7e5-4d13-be65-4251f4ac8d45 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.893322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.894004 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.894293 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.894612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.895353 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.896992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9420e3-8183-46cb-b588-94f6bcc1380e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.908827 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1406/2846] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:50.914779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.927167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bcd1a-ea93-449a-8d71-973ee1d7502a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.932357 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.933320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.933626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.933922 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] PATH_INFO: `/v3/projects/f4f022b0a8384911b255dfba6b87c2ce/users/a92fdf27378145e8a38b92637dd3c447/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.951545 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f022b0a8384911b255dfba6b87c2ce, user_id=a92fdf27378145e8a38b92637dd3c447, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.954400 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.971490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1441/2847] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:50 2022] PUT /identity/v3/projects/f4f022b0a8384911b255dfba6b87c2ce/users/a92fdf27378145e8a38b92637dd3c447/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:50.985067 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.985533 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.985867 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.034721 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] MFA Rules not processed for user `a92fdf27378145e8a38b92637dd3c447`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:51.036136 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:51.108818 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1407/2848] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:51.121848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:51.123818 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:51.195579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee33434-5633-4fc6-8150-01677dbd0393 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:51.201064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:51.202119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.202540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.202960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.213773 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:51.215341 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ab72-67a1-4e93-8d2e-1305484021d5 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:51.226301 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1442/2849] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:51.256207 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:51.326900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a3ee-eab2-4dff-80c2-1a7400649555 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:51.332467 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:51.333356 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.333636 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.333913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] PATH_INFO: `/v3/users/a92fdf27378145e8a38b92637dd3c447` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.335231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92fdf27378145e8a38b92637dd3c447)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:51.337190 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:51.393512 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92fdf27378145e8a38b92637dd3c447'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.406128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92fdf27378145e8a38b92637dd3c447'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.412379 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:51.426268 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92fdf27378145e8a38b92637dd3c447'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.427520 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92fdf27378145e8a38b92637dd3c447'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.428404 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92fdf27378145e8a38b92637dd3c447'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.446574 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1408/2850] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:51 2022] DELETE /identity/v3/users/a92fdf27378145e8a38b92637dd3c447 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:51.451618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.452164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.452164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.472562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:51.473785 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:51.479907 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:51.546726 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1443/2851] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:51.557841 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:51.559661 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:51.634986 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010339f-ebb6-4ef0-9803-da39d051fb7f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:51.639059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:51.639900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.640233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.640552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.655150 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:51.656705 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c62b8-9ec0-4d65-b7d9-79585fe41dc9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:51.668419 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1409/2852] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:52.014119 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.015878 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.033101 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2648f-9062-43f0-9ca5-d279f5ec4ea0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.038739 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.039784 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.040112 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.040430 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] PATH_INFO: `/v3/projects/f4f022b0a8384911b255dfba6b87c2ce` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.042240 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f022b0a8384911b255dfba6b87c2ce)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.047287 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.105235 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f022b0a8384911b255dfba6b87c2ce', 'request_id': 'req-42ae3d2b-1543-472e-9489-772cae4749dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.116955 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f022b0a8384911b255dfba6b87c2ce', 'request_id': 'req-42ae3d2b-1543-472e-9489-772cae4749dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.130813 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f022b0a8384911b255dfba6b87c2ce', 'request_id': 'req-42ae3d2b-1543-472e-9489-772cae4749dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.147713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] The token cache is being invalidate because project f4f022b0a8384911b255dfba6b87c2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:52.148203 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.149125 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-42ae3d2b-1543-472e-9489-772cae4749dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.150969 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1444/2853] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:52 2022] DELETE /identity/v3/projects/f4f022b0a8384911b255dfba6b87c2ce => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:52.156238 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.156590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.156805 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.175923 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:52.177148 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.245275 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1410/2854] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:52.309833 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.310124 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.310445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.330106 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:52.331394 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.339932 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1445/2855] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:52.345110 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.355785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18a1a3-b47d-4d25-bf0d-c7e4e21d528e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.363532 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.364331 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.364659 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.364976 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.365858 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:52.366222 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.367970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.381028 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1411/2856] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:50:52.388925 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.404409 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320480b-c3f5-4365-8afc-5c4d615cd491 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.412032 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.413093 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.413484 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.413846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.415245 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.418937 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.457206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1446/2857] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:52.461219 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.473163 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfc6cb-1bfc-456c-99b4-51e6a6f12d30 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.536097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.536793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.537080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.537357 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.538195 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.540235 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.557785 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:52.569886 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1412/2858] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:52.571133 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.581957 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7ca6-29fd-46e9-9a66-adabbe88aa3e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.586959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.587755 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.588046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.588330 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.589092 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.590801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.603646 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1447/2859] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:52.608036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.619760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a01b-dc1d-4057-a700-5a4b311cd8e8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.625134 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.625831 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.626119 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.626427 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.627196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.629005 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb059b19-7089-46d4-b1ba-fffadebac413 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.641182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1413/2860] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:52.647753 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.665892 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612a839-2b32-4581-b181-5fdd739a37b4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.673721 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.675291 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.675720 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.676121 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] PATH_INFO: `/v3/projects/19f81062b8064bca9d99d5307a638c8c/users/99fa41d7268d40918ed217fddd749950/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.699579 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f81062b8064bca9d99d5307a638c8c, user_id=99fa41d7268d40918ed217fddd749950, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.704674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.725769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1448/2861] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:52 2022] PUT /identity/v3/projects/19f81062b8064bca9d99d5307a638c8c/users/99fa41d7268d40918ed217fddd749950/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:52.741529 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.741820 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.742128 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.773350 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] MFA Rules not processed for user `99fa41d7268d40918ed217fddd749950`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:52.774633 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.846687 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1414/2862] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:52.859916 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.861914 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.907674 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:52.944838 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49198-b1a0-4756-818c-341197f36984 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.948647 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '527930459ed340c0b09ee6c6da770a21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.949408 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.949787 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.950132 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.960591 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.962110 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3263-edaf-44d4-9280-fcb1ff270d80 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.973657 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1449/2863] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:01.320107 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:01.427904 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9f306-a851-441f-8ddc-73f6318f5904 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:01.440055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:01.440966 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.441245 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.441621 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] PATH_INFO: `/v3/users/99fa41d7268d40918ed217fddd749950` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.443539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99fa41d7268d40918ed217fddd749950)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:01.445828 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:01.523396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99fa41d7268d40918ed217fddd749950'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.531900 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99fa41d7268d40918ed217fddd749950'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.546185 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99fa41d7268d40918ed217fddd749950'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.547694 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99fa41d7268d40918ed217fddd749950'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.548257 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99fa41d7268d40918ed217fddd749950'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.552753 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:01.567361 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1415/2864] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:01 2022] DELETE /identity/v3/users/99fa41d7268d40918ed217fddd749950 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:01.572173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.572732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.572732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.591064 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:01.592320 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:01.669668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1450/2865] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:01.683992 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:01.686888 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:01.785536 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eae7ec-823f-4e4d-bc66-afe621d44eb8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:01.789670 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:01.790433 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.790898 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.791273 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.802086 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:01.803540 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e76949-a674-40c5-b49d-8269187264c8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:01.815120 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1416/2866] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:02.232321 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.233807 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.248334 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473fc4f-4789-4091-a8d6-d51ae7ed21ab None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.254599 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.255564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.258457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.258457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] PATH_INFO: `/v3/projects/19f81062b8064bca9d99d5307a638c8c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.258869 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f81062b8064bca9d99d5307a638c8c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.263730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.347344 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f81062b8064bca9d99d5307a638c8c', 'request_id': 'req-0afe888a-5029-4285-b1e5-482a2f4d9bdb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.354254 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f81062b8064bca9d99d5307a638c8c', 'request_id': 'req-0afe888a-5029-4285-b1e5-482a2f4d9bdb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.365979 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f81062b8064bca9d99d5307a638c8c', 'request_id': 'req-0afe888a-5029-4285-b1e5-482a2f4d9bdb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.383621 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] The token cache is being invalidate because project 19f81062b8064bca9d99d5307a638c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:02.383939 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.384724 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0afe888a-5029-4285-b1e5-482a2f4d9bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.387370 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1451/2867] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:02 2022] DELETE /identity/v3/projects/19f81062b8064bca9d99d5307a638c8c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:02.394740 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.395577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.395577 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.414650 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.416013 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.489549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1417/2868] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:02.498783 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.499314 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.500352 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.535488 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.537723 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.552995 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1452/2869] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:02.558081 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.571196 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811eafa-bf86-4696-a927-599e0145dc14 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.576688 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.577494 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.577945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.578378 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.579336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:02.579742 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.581509 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.600663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1418/2870] 10.222.0.77 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:51:02.601913 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.614100 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f362e-2e74-4c0a-b18a-c565cb4aecc1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.619828 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.620559 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.620848 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.621164 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.621906 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.623959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.659122 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1453/2871] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:02.664206 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.677913 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f82cb-7743-4e10-a532-341019162f67 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.782214 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.783376 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.783865 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.784360 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.785617 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.788824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.817683 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1419/2872] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:02.822735 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.834232 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512928af-1f75-4208-a082-886a672401d5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.839220 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.839902 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.840186 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.840465 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.841255 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.842946 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.856705 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1454/2873] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:02.862031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.872967 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f145b-16f0-4d7b-b180-6113fed71f56 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.878622 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.879537 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.879956 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.880336 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.881249 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.883322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32278c86-fc02-4995-838c-94211778b79f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.894999 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1420/2874] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:02.900056 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.912086 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b95c-0549-407a-8c72-f8dba69ba891 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.917801 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.918778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.919062 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.919340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] PATH_INFO: `/v3/projects/b93b037768d74e83aa4ca3fd2725ee23/users/0638500c25cb43d9add4c4fdf8381898/roles/8569379203d74bee9bdcc46f1e112633` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.939960 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93b037768d74e83aa4ca3fd2725ee23, user_id=0638500c25cb43d9add4c4fdf8381898, role_id=8569379203d74bee9bdcc46f1e112633)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.942778 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.945508 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:02.963959 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1455/2875] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:02 2022] PUT /identity/v3/projects/b93b037768d74e83aa4ca3fd2725ee23/users/0638500c25cb43d9add4c4fdf8381898/roles/8569379203d74bee9bdcc46f1e112633 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:02.971269 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310ac3-822e-4978-805a-92750d384cec None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.971613 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310ac3-822e-4978-805a-92750d384cec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.971937 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310ac3-822e-4978-805a-92750d384cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.980605 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:02.995389 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8e310ac3-822e-4978-805a-92750d384cec None None] MFA Rules not processed for user `cf4ea7b7962d4647b8de9301728f33d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.997504 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e310ac3-822e-4978-805a-92750d384cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.077928 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1421/2876] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:03.090731 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.092618 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.173274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1c258-5db6-45cd-96d7-d4db9eac5f4e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.177713 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.178428 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.178889 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.179176 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.190504 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.192072 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fa52f-e17f-4d2a-8038-e6fbd18bf15f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.204065 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1456/2877] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:05.316125 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e310ac3-822e-4978-805a-92750d384cec None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.317940 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e310ac3-822e-4978-805a-92750d384cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.497059 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e310ac3-822e-4978-805a-92750d384cec None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.512824 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.514229 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.514955 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.515614 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.518872 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.548830 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.553303 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.572730 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1422/2878] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:09.582505 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.583485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.583485 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.626349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] MFA Rules not processed for user `0638500c25cb43d9add4c4fdf8381898`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:09.627809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.720809 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1457/2879] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:09.728371 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.743777 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a560f-0d81-46c3-acf8-acc16ea6a09f service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.749430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.751111 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.751590 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.752032 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.753180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.755791 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.795803 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1423/2880] 10.222.0.77 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:09.800261 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.812921 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6350773-5b6d-4851-bb7f-0a2b98606ee6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.902877 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.904023 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.904540 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.905047 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.906286 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.909971 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.950142 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1458/2881] 10.222.0.77 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:09 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:09.954492 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.974978 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd5f5-5437-45ae-9dec-3dbf5d2387fd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.985372 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.986714 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.987382 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.988180 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.989835 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.993311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.016200 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1424/2882] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:10.020769 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.032139 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04babe0f-530d-46ee-9ae3-fd0f641dec7a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.037671 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.038696 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.039080 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.039362 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] PATH_INFO: `/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca/users/e52d0ec93b0949a2865c2eb4b00caeba/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.056016 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:10.061445 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdc4067d89d482ca0e76b2e5752c6ca, user_id=e52d0ec93b0949a2865c2eb4b00caeba, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.068601 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.094799 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1459/2883] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:10 2022] PUT /identity/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca/users/e52d0ec93b0949a2865c2eb4b00caeba/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:10.101052 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.123719 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7768df-df00-4927-a0c8-8c654b9aad95 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.211143 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:10.220875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.221912 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.222401 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.222733 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.223718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.225583 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.254709 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1425/2884] 10.222.0.77 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:10.256704 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.277762 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c8645-2f92-45d4-99a1-bc89a237cbc4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.289709 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.290862 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.291224 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.291605 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] PATH_INFO: `/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca/users/e52d0ec93b0949a2865c2eb4b00caeba/roles/54d519e3caa64c32a9930f925d38c156` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.294895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdc4067d89d482ca0e76b2e5752c6ca, user_id=e52d0ec93b0949a2865c2eb4b00caeba, role_id=54d519e3caa64c32a9930f925d38c156)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.298654 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df63fa-30bf-4193-a5ea-16def65aa5f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.313077 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1460/2885] 10.222.0.77 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:10 2022] PUT /identity/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca/users/e52d0ec93b0949a2865c2eb4b00caeba/roles/54d519e3caa64c32a9930f925d38c156 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:12.659123 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.795072 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f4b9b-256c-4439-9323-c723fe4301a6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.899199 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.900908 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.901575 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.902183 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.905840 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.055104 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.059055 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.085168 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1426/2886] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:18.149389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873c18-b00c-4901-9c74-c6304406769d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:18.149920 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873c18-b00c-4901-9c74-c6304406769d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:18.150042 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73873c18-b00c-4901-9c74-c6304406769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:18.216849 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-73873c18-b00c-4901-9c74-c6304406769d None None] MFA Rules not processed for user `e52d0ec93b0949a2865c2eb4b00caeba`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:18.219167 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-73873c18-b00c-4901-9c74-c6304406769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:18.363401 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1461/2887] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:18.392568 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:18.394383 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:18.548182 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:18.614946 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a270d-2962-4de3-a687-d91c573c5a41 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:18.629155 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:18.629927 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:18.630282 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:18.630663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:18.757710 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:18.759458 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:18.772790 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1427/2888] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:21.191626 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:21.192557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:21.192557 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:21.224577 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] MFA Rules not processed for user `0638500c25cb43d9add4c4fdf8381898`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:21.227740 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:21.240779 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1462/2889] 10.222.0.77 () {58 vars in 957 bytes} [Mon Jul 18 16:51:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:25.481191 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:25.492662 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:25.519048 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e3353-ed41-488e-ad98-0def6289f757 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:25.524516 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:25.525612 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:25.525970 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:25.526343 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] PATH_INFO: `/v3/users/6c7e0694c5ab46a4adef7a1f7b1fed4c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:25.527958 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c7e0694c5ab46a4adef7a1f7b1fed4c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:25.531080 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:25.594581 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c7e0694c5ab46a4adef7a1f7b1fed4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:25.604036 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c7e0694c5ab46a4adef7a1f7b1fed4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:25.621311 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c7e0694c5ab46a4adef7a1f7b1fed4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:25.622227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c7e0694c5ab46a4adef7a1f7b1fed4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:25.622983 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c7e0694c5ab46a4adef7a1f7b1fed4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:25.646089 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1428/2890] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:25 2022] DELETE /identity/v3/users/6c7e0694c5ab46a4adef7a1f7b1fed4c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:25.902570 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:25.926611 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a900f1-c15c-44c0-b68d-c805e7524e79 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.025247 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.026048 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.026349 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.026659 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] PATH_INFO: `/v3/projects/edffd84b958d4e2a880fae7c4709d36d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.027934 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] RBAC: Authorizing `identity:delete_project(project_id=edffd84b958d4e2a880fae7c4709d36d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.029898 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.084764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edffd84b958d4e2a880fae7c4709d36d', 'request_id': 'req-b093fb29-bcc2-4099-ba33-623d3822457e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.092398 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edffd84b958d4e2a880fae7c4709d36d', 'request_id': 'req-b093fb29-bcc2-4099-ba33-623d3822457e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.097015 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:26.105123 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edffd84b958d4e2a880fae7c4709d36d', 'request_id': 'req-b093fb29-bcc2-4099-ba33-623d3822457e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.125513 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] The token cache is being invalidate because project edffd84b958d4e2a880fae7c4709d36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:26.125806 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.126562 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.128300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1463/2891] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:25 2022] DELETE /identity/v3/projects/edffd84b958d4e2a880fae7c4709d36d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:26.133210 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.135487 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:26.161227 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f5652-e269-4d29-a047-e2cfec4ad41d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.234646 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.235413 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.235701 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.235980 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] PATH_INFO: `/v3/users/d2a021c05c0744eaa6a282b1e37292fa` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.237287 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a021c05c0744eaa6a282b1e37292fa)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.239342 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.252326 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:26.308843 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a021c05c0744eaa6a282b1e37292fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.318957 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a021c05c0744eaa6a282b1e37292fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.334334 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a021c05c0744eaa6a282b1e37292fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.335789 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a021c05c0744eaa6a282b1e37292fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.336910 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a021c05c0744eaa6a282b1e37292fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.355179 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1429/2892] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:26 2022] DELETE /identity/v3/users/d2a021c05c0744eaa6a282b1e37292fa => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:26.590393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.593393 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:26.617141 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fb29-bcc2-4099-ba33-623d3822457e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.698246 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.699173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.699457 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.699738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] PATH_INFO: `/v3/projects/774a83196bf843c4aedd7519b0a44dca` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.701075 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=774a83196bf843c4aedd7519b0a44dca)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.703187 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.770970 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774a83196bf843c4aedd7519b0a44dca', 'request_id': 'req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.778028 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774a83196bf843c4aedd7519b0a44dca', 'request_id': 'req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.787566 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774a83196bf843c4aedd7519b0a44dca', 'request_id': 'req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.800730 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] The token cache is being invalidate because project 774a83196bf843c4aedd7519b0a44dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:26.801057 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.801714 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.803738 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1464/2893] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:26 2022] DELETE /identity/v3/projects/774a83196bf843c4aedd7519b0a44dca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:36.131159 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.132760 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.197924 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d9-b8fe-4917-9da3-17ca92adf208 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.256941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.257746 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.258122 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.258558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.260538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.271481 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.273001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.283729 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1430/2894] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:36.797050 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=55,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:36.804610 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.808046 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.911435 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cb56-acbb-4bd3-abf1-9fa9c658d8cb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.917174 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.917945 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.918234 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.918564 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.920732 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.936340 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.938650 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.950228 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1465/2895] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:46.278783 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=10,keystone:SELECT=68,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:46.945389 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=32 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:49.619370 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:49.620987 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:49.729396 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c7c-9de6-47d8-ac74-c1deae326a3f service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:49.735573 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:49.736242 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:49.736526 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:49.736806 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:49.739552 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:49.752765 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:49.754398 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:49.767147 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1431/2896] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:52.751144 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.752764 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.765846 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47871ee0-3db5-4cc4-815c-7c5964f6c676 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.769558 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.770483 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.770895 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.771226 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.773274 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.848030 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.849525 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.861300 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1466/2897] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:59.761400 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=34 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:02.855912 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=30 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:06.087715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:06.089430 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:06.104772 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf139a8-38ce-4c96-bdf3-70bddb836429 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:06.110453 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:06.111118 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:06.111405 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:06.111684 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:06.113558 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:06.128252 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:06.129945 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:06.141455 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1432/2898] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:53:16.136732 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:04.134680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:04.143814 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993c2b2-7ee7-45eb-8e89-f245aa284be4 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:04.147673 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:04.148335 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:04.148622 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:04.148897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:04.151910 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:04.162759 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:04.164225 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:04.174206 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1467/2899] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:05.272718 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:05.274345 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:05.286085 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d0672-6729-45c1-ab02-79b2949a4c29 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:05.289875 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:05.290563 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:05.290856 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:05.291138 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:05.293131 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:05.308897 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:05.311077 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:05.323934 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1433/2900] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:14.169151 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:15.318518 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:08.727653 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:08.737379 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda49ab-b052-4cb1-856e-86fe85d38436 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:08.741384 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:08.742060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:08.742348 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:08.742668 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:08.744651 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:08.768734 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:08.770236 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:08.780527 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1468/2901] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:18.775595 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=10 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:36.719380 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:36.760965 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225f0fe-f7c8-423e-8ee9-5dbd31ba3482 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:36.764839 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:36.765486 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:36.765775 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:36.766053 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:36.767917 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:36.803240 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:36.805097 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:36.817352 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1434/2902] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:37.997406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:38.036944 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7fbb4-f925-4c96-a7ca-cc5eaed17847 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:38.040854 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747a2eb4850417d8d02a2b0a8d0cff7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:38.041499 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:38.041786 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:38.042060 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:38.044209 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:38.058804 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:38.060173 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:38.071500 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1469/2903] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:38.869061 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:38.870634 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:38.886057 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507579ee-0eff-4fcd-948f-dd3270dc8854 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:38.891728 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd1cd1cd83fa43cab0dcbd1698c715fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:38.892375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:38.892663 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:38.892941 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:38.894744 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:38.909802 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:38.911428 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:38.922711 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1435/2904] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:43.648903 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:43.662956 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d510-5d46-4cae-80dc-f455be13af3d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:43.669260 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:43.669434 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:43.669588 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:43.669868 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:43.671847 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:43.685216 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:43.686927 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:43.698127 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1470/2905] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:46.640538 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:46.653127 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:46.757821 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafebfa1-3021-4941-af61-e814651bd344 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:46.762375 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:46.763263 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:46.763539 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:46.763807 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] PATH_INFO: `/v3/users/0638500c25cb43d9add4c4fdf8381898` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:46.764985 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0638500c25cb43d9add4c4fdf8381898)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:46.766979 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:46.817031 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0638500c25cb43d9add4c4fdf8381898'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:46.823787 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0638500c25cb43d9add4c4fdf8381898'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:46.828178 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:46.836778 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0638500c25cb43d9add4c4fdf8381898'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:46.837491 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0638500c25cb43d9add4c4fdf8381898'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:46.838082 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0638500c25cb43d9add4c4fdf8381898'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:46.851445 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1436/2906] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:55:46 2022] DELETE /identity/v3/users/0638500c25cb43d9add4c4fdf8381898 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:47.026498 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:47.039006 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64c91-0579-4e49-9c71-e93df32c1ed0 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:47.097034 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:47.097897 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:47.098183 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:47.098490 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] PATH_INFO: `/v3/projects/b93b037768d74e83aa4ca3fd2725ee23` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:47.099680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93b037768d74e83aa4ca3fd2725ee23)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:47.101658 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:47.148229 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93b037768d74e83aa4ca3fd2725ee23', 'request_id': 'req-35357ed3-d74a-439f-94bd-10359a66b95b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.154157 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93b037768d74e83aa4ca3fd2725ee23', 'request_id': 'req-35357ed3-d74a-439f-94bd-10359a66b95b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.162326 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93b037768d74e83aa4ca3fd2725ee23', 'request_id': 'req-35357ed3-d74a-439f-94bd-10359a66b95b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.173049 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] The token cache is being invalidate because project b93b037768d74e83aa4ca3fd2725ee23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:55:47.173343 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.174555 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.176118 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1471/2907] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:55:47 2022] DELETE /identity/v3/projects/b93b037768d74e83aa4ca3fd2725ee23 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:47.180322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:47.181745 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:47.195297 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c228d-6459-4d4e-8d95-7e738ed12a0e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:47.252233 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:47.253135 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:47.253416 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:47.253691 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] PATH_INFO: `/v3/users/e52d0ec93b0949a2865c2eb4b00caeba` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:47.254959 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52d0ec93b0949a2865c2eb4b00caeba)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:47.256971 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:47.306549 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52d0ec93b0949a2865c2eb4b00caeba'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.313599 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52d0ec93b0949a2865c2eb4b00caeba'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.325322 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52d0ec93b0949a2865c2eb4b00caeba'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.326001 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52d0ec93b0949a2865c2eb4b00caeba'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.326643 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52d0ec93b0949a2865c2eb4b00caeba'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.339522 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1437/2908] 10.222.0.77 () {58 vars in 1223 bytes} [Mon Jul 18 16:55:47 2022] DELETE /identity/v3/users/e52d0ec93b0949a2865c2eb4b00caeba => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:47.515153 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:47.517066 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:47.529585 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ed3-d74a-439f-94bd-10359a66b95b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:47.565965 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:47.591532 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf4ea7b7962d4647b8de9301728f33d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c66249a1b75d4939b69e5011863c8709', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:47.592406 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:47.592693 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:47.592975 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] PATH_INFO: `/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:47.594196 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcdc4067d89d482ca0e76b2e5752c6ca)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:47.596145 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:47.642924 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcdc4067d89d482ca0e76b2e5752c6ca', 'request_id': 'req-96e60c7a-c451-49a0-be15-7b10f54bc83f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.649296 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcdc4067d89d482ca0e76b2e5752c6ca', 'request_id': 'req-96e60c7a-c451-49a0-be15-7b10f54bc83f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.656031 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcdc4067d89d482ca0e76b2e5752c6ca', 'request_id': 'req-96e60c7a-c451-49a0-be15-7b10f54bc83f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.666360 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] The token cache is being invalidate because project dcdc4067d89d482ca0e76b2e5752c6ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:55:47.666680 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.667332 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-96e60c7a-c451-49a0-be15-7b10f54bc83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:47.669155 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1472/2909] 10.222.0.77 () {58 vars in 1232 bytes} [Mon Jul 18 16:55:47 2022] DELETE /identity/v3/projects/dcdc4067d89d482ca0e76b2e5752c6ca => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:57.336071 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=13,keystone:SELECT=59,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:57.663342 dv-cinder-837543-6-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:58:14.024651 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:14.026231 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:14.140292 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e18dd-2ad3-448c-967a-60892cbe6f27 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:14.211049 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52152797f2784be389bfae5dd38ef5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e91ceaccc1f4cc5a51c17a9216967ea', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:14.211726 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:14.212015 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:14.212295 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:14.214308 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:14.228793 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:14.231715 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef7a87-1ed8-48b1-9528-4d9c55bc4692 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:14.244034 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1438/2910] 10.222.0.77 () {60 vars in 1365 bytes} [Mon Jul 18 16:58:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:58:24.238950 dv-cinder-837543-6-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=67 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}